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Abstract 

Correlations in spike-train ensembles can seriously impair the encoding of information by their 
spatio-temporal structure. An inevitable source of correlation in finite neural networks is com- 
mon presynaptic input to pairs of neurons. Recent studies demonstrate that spike correlations 
in recurrent neural networks are considerably smaller than expected based on the amount 
of shared presynaptic input. Here, we explain this observation by means of a linear network 
model and simulations of networks of leaky integrate-and-fire neurons. We show that inhibitory 
feedback efficiently suppresses pairwise correlations and, hence, population-rate fluctuations, 
thereby assigning inhibitory neurons the new role of active decorrelation. We quantify this 
decorrelation by comparing the responses of the intact recurrent network (feedback system) 
and systems where the statistics of the feedback channel is perturbed (feedforward system). 
Manipulations of the feedback statistics can lead to a significant increase in the power and co- 
herence of the population response. In particular, neglecting correlations within the ensemble of 
feedback channels or between the external stimulus and the feedback amplifies population-rate 
fluctuations by orders of magnitude. The fluctuation suppression in homogeneous inhibitory 
networks is explained by a negative feedback loop in the one-dimensional dynamics of the 
compound activity. Similarly, a change of coordinates exposes an effective negative feedback 
loop in the compound dynamics of stable excitatory-inhibitory networks. The suppression of 
input correlations in finite networks is explained by the population averaged correlations in 
the linear network model: In purely inhibitory networks, shared-input correlations are canceled 
by negative spike-train correlations. In excitatory-inhibitory networks, spike-train correlations 
are typically positive. Here, the suppression of input correlations is not a result of the mere 
existence of correlations between excitatory (E) and inhibitory (I) neurons, but a consequence 
of a particular structure of correlations among the three possible pairings (EE, El, II). 

Author summary 

The spatio-temporal activity pattern generated by a recurrent neuronal network can provide a 
rich dynamical basis which allows readout neurons to generate a variety of responses by tuning 
the synaptic weights of their inputs. The repertoire of possible responses and the response 
reliability become maximal if the spike trains of individual neurons are uncorrelated. Spike-train 
correlations in cortical networks can indeed be very small, even for neighboring neurons. This 
seems to be at odds with the finding that neighboring neurons receive a considerable fraction 
of inputs from identical presynaptic sources constituting an inevitable source of correlation. 
In this article, we show that inhibitory feedback, abundant in biological neuronal networks, 
actively suppresses correlations. The mechanism is generic: It does not depend on the details 
of the network nodes and decorrelates networks composed of excitatory and inhibitory neurons 
as well as purely inhibitory networks. For the case of the leaky integrate-and-fire model, we 
derive the correlation structure analytically. The new toolbox of formal linearization and a 
basis transformation exposing the feedback component is applicable to a range of biological 
systems. We confirm our analytical results by direct simulations. 

1 Introduction 

Neurons generate signals by weighting and combining input spike trains from presynaptic 
neuron populations. The number of possible signals which can be read out this way from a 
given spike-train ensemble is maximal if these spike trains span an orthogonal basis, i.e. if they 
are uncorrelated [77]. If they are correlated, the amount of information which can be encoded 
in the spatio-temporal structure of these spike trains is limited. In addition, correlations impair 
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the ability of readout neurons to decode information reliably in the presence of noise. This is 
often discussed in the context of rate coding: for TV uncorrected spike trains, the signal-to- 
noise ratio of the compound spike-count signal can be enhanced by increasing the population 
size N. In the presence of correlations, however, the signal-to-noise ratio is bounded [85, 63]. 
The same reasoning holds for any other linear combination of spike trains, also for those where 
exact spike timing matters (for example for the coding scheme presented in [77]). Thus, the 
robustness of neuronal responses against noise critically depends on the level of correlated 
activity within the presynaptic neuron population. 

Several studies suggested that correlated neural activity could be beneficial for information 
processing: Spike-train correlations can modulate the gain of postsynaptic neurons and thereby 
constitute a gating mechanism (for a review, see [62]). Coherent spiking activity might serve 
as a means to bind elementary representations into more complex objects [82, 6]. Information 
represented by correlated firing can be reliably sustained and propagated through feedforward 
subnetworks ('synfire chains'; [1, 17]). Whether correlated firing has to be considered favorable 
or not largely depends on the underlying hypothesis, the type of correlation (e.g. the time scale 
or the affected frequency band) or which subpopulations of neurons are involved. Most ideas 
suggesting a functional benefit of correlated activity rely on the existence of an asynchronous 
'ground state'. Spontaneously emerging correlations, i.e. correlations which are not triggered 
by internal or external events, would impose a serious challenge to many of these hypotheses. 
Functionally relevant synfire activity, for example, cannot be guaranteed in the presence of 
correlated background input from the embedding network [73]. It is therefore-from several 
perspectives-important to understand the origin of uncorrected activity in neural networks. 

It has recently been shown that spike trains of neighboring cortical neurons can indeed 
be highly uncorrected [18]. Similar results have been obtained in several theoretical studies 
[81, 9, 22, 74, 37, 30, 57]. From an anatomical point of view, this observation is puzzling: 
in general, neurons in finite networks share a certain fraction of their presynaptic sources. 
In particular for neighboring neurons, the overlap between presynaptic neuron populations is 
expected to be substantial. This feedforward picture suggests that such presynaptic overlap 
gives rise to correlated synaptic input and, in turn, to correlated response spike trains. 

A number of theoretical studies showed that shared-input correlations are only weakly 
transferred to the output side as a consequence of the nonlinearity of the spike-generation 
dynamics [70, 71, 45, 16, 37]. Unreliable spike transmission due to synaptic failure can further 
suppress the correlation gain [60]. In [73], we demonstrated that spike-train correlations in 
finite-size recurrent networks are even smaller than predicted by the low correlation gain of 
pairs of neurons with nonlinear spike-generation dynamics. We concluded that this suppres- 
sion of correlations must be a result of the recurrent network dynamics. In this article, we 
compare correlations observed in feedforward networks to correlations measured in systems 
with an intact feedback loop. We refer to the reduction of correlations in the presence of 
feedback as "decorrelation". Different mechanisms underlying such a dynamical decorrelation 
have been suggested in the recent past. Asynchronous states in recurrent neural networks 
are often attributed to chaotic dynamics [4, 44]. In fact, networks of nonlinear units with 
random connectivity and balanced excitation and inhibition typically exhibit chaos [81, 33]. 
The high sensitivity to noise may however question the functional relevance of such systems 
([38, 32]; cf., however, [75]). [84] and [32] demonstrated that asynchronous irregular firing 
can also emerge in networks with stable dynamics. Employing an analytical framework of 
correlations in recurrent networks of binary neurons [24], the balance of excitation and inhibi- 
tion has recently been proposed as another decorrelation mechanism [57]: In large networks, 
fluctuations of excitation and inhibition are in phase. Positive correlations between excitatory 
and inhibitory input spike trains lead to a negative component in the net input correlation 
which can compensate positive correlations caused by shared input. 
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In the present study, we demonstrate that dynamical decorrelation is a fundamental phe- 
nomenon in recurrent systems with negative feedback. We show that negative feedback alone 
is sufficient to efficiently suppress correlations. Even in purely inhibitory networks, shared-input 
correlations are compensated by feedback. A balance of excitation and inhibition is thus not 
required. The underlying mechanism can be understood by means of a simple linear model. 
This simplifies the theory and helps to gain intuition, but it also confirms that low correlations 
can emerge in recurrent networks with stable, non-chaotic dynamics. 

The suppression of pairwise spike-train correlations by inhibitory feedback is reflected in 
a reduction of population-rate fluctuations. The main effect described in this article can 
therefore be understood by studying the dynamics of the macroscopic population activity 
(Sec. 2.1-Sec. 2.3). This approach leads to a simple mathematical description and emphasizes 
that the described decorrelation mechanism is a general phenomenon which may occur not only 
in neural networks but also in other (biological) systems with inhibitory feedback. In Sec. 2.1, 
we first illustrate the decorrelation effect for random networks of TV leaky integrate-and-fire 
(LIF) neurons with inhibitory or excitatory-inhibitory coupling. By means of simulations, we 
show that low-frequency spike-train correlations, and, hence, population-rate fluctuations are 
considerably smaller than expected given the amount of shared input. As shown in Sec. 2.2, the 
suppression of population-rate fluctuations by inhibitory feedback can readily be understood 
in the framework of a simple one-dimensional linear model with negative feedback. Sec. 2.3 
extends this result to a two-population system with excitatory-inhibitory coupling. Here, a 
simple coordinate transform exposes the inherent negative feedback loop as the underlying 
cause of the fluctuation suppression in inhibition-dominated networks. The population-rate 
models used in Sec. 2.2 and Sec. 2.3 are sufficient to understand the basic mechanism under- 
lying the decorrelation. They do, however, not describe how feedback in cortical networks 
affects the detailed structure of pairwise correlations. In Sec. 2.4, we therefore compute self- 
consistent population averaged correlations for a random network of TV linear excitatory and 
inhibitory neurons. By determining the parameters of the linear network analytically from the 
LIF model, we show that the predictions of the linear model are — for a wide and realistic range 
of parameters — in excellent agreement with the results of the LIF network model. In Sec. 2.5, 
we make clear that the active decorrelation in random LIF networks relies on the feedback 
of the (sub)population averaged activity but not on the precise microscopic structure of the 
feedback signal. In Sec. 3, we discuss the consequences of this work in a broader context and 
point out limitations and possible extensions of the presented theory. Sec. 4 contains details 
on the LIF network model, the derivation of the linear model from the LIF dynamics and the 
derivation of population-rate spectra and population averaged correlations in the framework 
of the linear model. It is meant as a supplement; the basic ideas and the main results can be 
extracted from Sec. 2. 

2 Results 

In a recurrent neural network of size N, each neuron i e [1, iV] receives in general inputs 
from two different types of sources: External inputs <^(£) representing the sum of afferents 
from other brain areas, and local inputs resulting from the recurrent connectivity within the 
network. Depending on their origin, external inputs & and t;j to different neurons % and j 
can be correlated or not. Throughout this manuscript, we ignore correlations between these 
external sources, thereby ensuring that correlations within the network activity arise from the 
local connectivity alone and are not imposed by external inputs [57]. The local inputs feed the 
network's spiking activity s(t) = (si(t), • • • ,s N (t)) T back to the network (we refer to spike 
train Si(t), the zth component of the column vector s(t) [the superscript "T" denotes the 
transpose], as a sum over delta-functions centered at the spike times t\\ Si(t) = ^^(t — tf); 
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the abstract quantity 'spike train' can be considered as being derived from the observable 
quantity 'spike count' nf t (t) 1 the number of spikes occurring in the time interval [i, t + At), 
by taking the limit At —> 0: Si(t) = lim A ^o *(*))■ The structure and weighting of this 
feedback can be described by the network's connectivity matrix J (see Fig. 1A). In a finite 
network, the local connectivity typically gives rise to overlapping presynaptic populations: in 
a random (Erdos-Renyi) network with connection probability e, for example, each pair of 
postsynaptic neurons shares, on average, e 2 N presynaptic sources. For a network size of, say, 
TV = 10 4 and a connection probability e = 0.1, this corresponds to a fairly large number of 
100 identical inputs. For other network structures, the amount of shared input may be smaller 
or larger. Due to this presynaptic overlap, each pair of neurons receives, to some extent, 
correlated input (even if the external inputs are uncorrelated). One might therefore expect 
that the network responses si(t), . . . , s N (t) are correlated as well. In this article, we show 
that, in the presence of negative feedback, the effect of shared input caused by the structure 
of the network is compensated by its recurrent dynamics. 



2.1 Suppression of population-rate fluctuations in LIF networks 



To illustrate the effect of shared input and its suppression by the recurrent dynamics, we com- 
pare the spike response s(t) = (si(t), • • • ,s N (t)) T of a recurrent random network (feedback 
scenario; Fig. 1A,C,E) of TV LIF neurons to the case where the feedback is cut and replaced by 
a spike-train ensemble q(t) = (qi(t), . . . , qN(t)) T , modeled by TV independent realizations of a 
stationary Poisson point process (feedforward scenario] Fig. 1B,D,F). The rate of this Poisson 
process is identical to the time and population averaged firing rate in the intact recurrent 
system. In both the feedback and the feedforward case, the (local) presynaptic spike trains are 
fed to the postsynaptic population according to the same connectivity matrix J. Therefore, 
not only the in-degrees and the synaptic weights but also the shared-input statistics are exactly 
identical. 

For realistic size TV and connectivity e, asynchronous states of random neural networks 
[11, 9] exhibit spike-train correlations which are small but not zero (compare raster displays 
in Fig. 1C and D; see also [37]). Although the presynaptic spike trains are, by construction, 
independent in the feedforward case (Fig. ID), the resulting response correlations, and, hence, 
the population-rate fluctuations, are substantially stronger than those observed in the feedback 
scenario (compare Fig. IF and E). In other words: A theory which is exclusively based on 
the amount of shared input but neglects the details of the presynaptic spike-train statistics 
can significantly overestimate correlations and population-rate fluctuations in recurrent neural 
networks. 

The same effect can be observed in LIF networks with both purely inhibitory and mixed 
excitatory-inhibitory coupling (Fig. 2). To demonstrate this quantitatively, we focus on the 
fluctuations of the population averaged activity s(t) = Its power-spectrum 

(or auto-correlation, in the time domain) 



Css(u) = \S(u)f = \Z[s(t)] M| 2 = ^ 



N 



N 



(1) 



is determined both by the power-spectra (auto-correlations) Ai(uS) = |^(cj)| 2 of the indi- 
vidual spike trains and the cross-spectra (cross-correlations) Cij(u) = Si(oj)Sj(u)* (i ^ j) 
of pairs of spike trains (throughout the article, we use capital letters to represent quantities 

in frequency [Fourier] space; Sk(w) = $[sk(t)] (cj) = J dt s k (t)e~ luJt represents the Fourier 

transform of the spike train s k (t)). We observe that the spike-train power-spectra Ai(uS) (and 
auto-correlations) are barely distinguishable in the feedback and in the feedforward case (not 
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shown here; the main features of the spike-train auto-correlation are determined by the average 
single-neuron firing rate and the refractory mechanism; both are identical in the feedback and 
the feedforward scenario). The differences in the population-rate spectra Css(w) are therefore 
essentially due to differences in the spike-train cross-spectra Cij{uS). In other words, the fluc- 
tuations in the population activity serve as a measure of pairwise spike-train correlations [25]: 
small (large) population averaged spike-train correlations are accompanied by small (large) 
fluctuations in the population rate (see lower panels in Fig. 1C-F). The power-spectra Css{w) 
of the population averaged activity reveal a feedback-induced suppression of the population- 
rate variance at low frequencies up to several tens of Hertz. For the examples shown in Fig. 2, 
this suppression spans more than three orders of magnitude for the inhibitory and more than 
one order of magnitude for the excitatory-inhibitory network. 

The suppression of low-frequency fluctuations does not critically depend on the details of 
the network model. As shown in Fig. 2, it can, for example, be observed for both networks 
with zero rise-time synapses (5-shaped synaptic currents) and short delays and for networks 
with delayed low-pass filtering synapses (a-shaped synaptic currents). In the latter case, the 
suppression of fluctuations is slightly more restricted to lower frequencies (< 10 Hz). Here, 
the fluctuation suppression is however similarly pronounced as in networks with instantaneous 
synapses. 

In Fig. 2C,D, the power-spectra of the population activity converge to the mean firing rate 
at high frequencies. This indicates that the spike trains are uncorrected on short time scales. 
For instantaneous 5-synapses, neurons exhibit an immediate response to excitatory input spikes 
[28, 58]. This fast response causes spike-train correlations on short time scales. Hence, the 
compound power at high frequencies is increased. In a recurrent system, this effect is amplified 
by reverberating simultaneous excitatory spikes. Therefore, the high-frequency power of the 
compound activity is larger in the feedback case (Fig. 2B). Note that this high-frequency effect 
is absent in networks with more realistic low-pass filtering synapses (Fig. 2C,D) and in purely 
inhibitory networks (Fig. 2A). 

Synaptic delays and slow synapses can promote oscillatory modes in certain frequency 
bands [11, 9], thereby leading to peaks in the population-rate spectra in the feedback scenario 
which exceed the power in the feedforward case (see peaks at ~ 25 Hz in Fig. 2C,D). Note 
that, in the feedforward case, the local input was replaced by a stationary Poisson process, 
whereas in the recurrent network (feedback case) the presynaptic spike trains exhibit oscillatory 
modes. By replacing the feedback by an inhomogeneous Poisson process with a time dependent 
intensity which is identical to the population rate in the recurrent network, we found that these 
oscillatory modes are neither suppressed nor amplified by the recurrent dynamics, i.e. the peaks 
in the resulting power-spectra have the same amplitude in the feedback and in the feedforward 
case (data not shown here). At low frequencies, however, the results are identical to those 
obtained by replacing the feedback by a homogeneous Poisson process (i.e. to those shown in 
Fig. 2; see Sec. 2.5). In the present study, we mainly focus on these low-frequency effects. 

The observation that the suppression of low-frequency fluctuations is particularly pro- 
nounced in networks with purely inhibitory coupling indicates that inhibitory feedback may 
play a key role for the underlying mechanism. In the following subsection, we will demonstrate 
by means of a one-dimensional linear population model that, indeed, negative feedback alone 
leads to an efficient fluctuation suppression. 

2.2 Suppression of population-activity fluctuations by negative feed- 
back 

Average pairwise correlations can be extracted from the spectrum (1) of the compound activity, 
provided the single spike-train statistics (auto-correlations) is known (see previous section). 
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Figure 1. Spiking activity in excitatory-inhibitory LIF networks with intact (left column; feed- 
back scenario) and opened feedback loop (right column; feedforward scenario). A,B: Network 
sketches for the feedback (A) and feedforward scenario (B). C,D: Spiking activity (top panels) 
and population averaged firing rate (bottom panels) of the local presynaptic populations. E,F: 
Response spiking activity (top panels) and population averaged response rate (bottom panels). 
In the top panels of C-F, each pixel depicts the number of spikes (gray coded) of a subpopula- 
tion of 250 neurons in a 10 ms time interval. In both the feedback and the feedforward scenario, 
the neuron population {1, . . . , TV} is driven by the same realization £(t) = • • • , 6v(^)) T 

of an uncorrelated white-noise ensemble; local input is fed to the population through the same 
connectivity matrix J. The in-degrees, the synaptic weights and the shared-input statistics are 
thus exactly identical in the two scenarios. In the feedback case (A), local presynaptic spike- 
trains are provided by the network's response s(t) = (si(t), . • • , s N (t)) T , i.e. the pre- (C) and 
postsynaptic spike-train ensembles (E) are identical. In the feedforward scenario (B), the local 
presynaptic spike-train population is replaced by an ensemble of TV independent realizations 
q(t) = (qi(t), . . . ,q N (t)) of a Poisson point process (D). Its rate is identical to the time- 
and population-averaged firing rate in the feedback case. See Tab. 1 and Tab. 2 for details on 
network models and parameters. 



As the single spike-train statistics is identical in the feedback and in the feedforward sce- 
nario, the mechanism underlying the decorrelation in recurrent networks can be understood by 
studying the dynamics of the population averaged activity. In this and in the next subsection 
(Sec. 2.3), we will consider the linearized dynamics of random networks composed of homo- 
geneous subpopulations of LIF neurons. The high-dimensional dynamics of such systems can 
be reduced to low-dimensional models describing the dynamics of the compound activity (for 
details, see Sec. 4.2). Note that this reduction is exact for networks with homogeneous out- 
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inhibitory coupling, 5-synapses, d = 0.1ms p exc.-inh. coupling, 5-synapses, d = 0.1ms 

I 1 I 1 J F 1 I 1 I 1 r 




10° 10 1 10 2 10 3 10° 10 1 10 2 10 3 

frequency (Hz) frequency (Hz) 

Figure 2. Suppression of low-frequency fluctuations in recurrent LIF networks with purely 
inhibitory (A,C) and mixed excitatory-inhibitory coupling (B,D) for instantaneous synapses 
with delay d = 0.1 ms (A,B) and low-pass synapses with d = 1 ms (C,D). Power-spectra NCss 
of population rates s(t) for the feedback (black) and the feedforward case (gray; cf. Fig. 1). 
See Tab. 1 and Tab. 2 for details on network models and parameters. In C and D, local 
synaptic inputs are modeled as currents Ii(t) = Jij^2ipsc(t — t\ — d) with a-function 
shaped kernel psc(i) = etr~ x exp(— t/r s )Q(i) with time constant r s = 10 ms (B(-) denotes 
Heaviside function). (Excitatory) Synaptic weights are set to J = 1 pA (see Tab. 1 for details). 
Simulation time T = 100 s. Single-trial spectra smoothed by moving average (frame size 1 Hz). 



degree (number of outgoing connections). For the networks studied here (random networks 
with homogeneous in-degree), it serves as a sufficient approximation (in a network of size TV 
where each connection is randomly and independently realized with probability e [Erdos-Renyi 
graph], the [binomial] in- and out-degree distributions become very sharp for large TV [relative 
to the mean in /out-degree]; both in- and out-degree are therefore approximately constant 
across the population of neurons). In this subsection, we will first study networks with purely 
inhibitory coupling. In Sec. 2.3, we will investigate the effect of mixed excitatory-inhibitory 
connectivity. 

Consider a random network of TV identical neurons with connection probability e. Each 
neuron i — 1, . . . , N receives K = eN randomly chosen inputs from the local network with 
synaptic weights —J. In addition, the neurons are driven by external uncorrected Gaussian 
white noise &(£) with amplitude rj, i.e. E t [£i(t)] = and E t [^i(t)Q(t + r)] = 5ijT] 2 5(r). For 
small input fluctuations, the network dynamics can be linearized. This linearization is based on 
the averaged response of a single neuron to an incoming spike and describes the activity of an 
individual neuron i by an abstract fluctuating quantity ri(t) which is defined such that within 
the linear approximation its auto- and cross-correlations fulfill the same linearized equation as 
the spiking model in the low-frequency limit. Consequently, also the low-frequency fluctuations 
of the population spike rate are captured correctly by the reduced model up to linear order. 
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This approach is equivalent to the treatment of finite-size fluctuations in spiking networks 
(see, e.g. [11]). For details see Sec. 4.2. For large TV, the population averaged activity 
r(t) = [ri{t)\ = TV -1 J2iLi r i(t) can hence be described by a one-dimensional linear system 

r(t) = ([-wr + x] *ti)(t) (2) 

with linear kernel h(t), effective coupling strength w = Kw and the population averaged 
noise x(t) = [xi(t)] (see Sec. 4.2 and Fig. 3B). The coupling strength w represents the 
integrated linear response of the neuron population to a small perturbation in the input rate 
of a single presynaptic neuron. For a population of LIF neurons, its relation to the synaptic 
weight J (PSP amplitude) is derived in Sec. 4.2 and Sec. 4.3. The normalized kernel h(t) (with 
Jq 00 At h(t) = 1) captures the time course of the linear response. It is determined by the single- 
neuron properties (e.g. the spike-initiation dynamics [20, 48]), the properties of the synapses 
(e.g. synaptic weights and time constants [10, 50]) and the properties of the input (e.g. exci- 
tatory vs. inhibitory input [55]). For many real and model neurons, the linear population-rate 
response exhibits low-pass characteristics [35, 36, 8, 7, 34, 22, 10, 40, 19, 20, 48, 55, 50, 58]. 
For illustration (Fig. 3), we consider a lst-order low-pass filter, i.e. an exponential impulse 
response h{t) = r" 1 exp(— t/r)Q(t) with time constant r (cutoff frequency f c = (27tt) -1 ; 
see Fig. 3A, light gray curve in E). The results of our analysis are however independent of 
the choice of the kernel h(t). The auto-correlation E t [x(t)x(t + r)] = p 2 S(r) of the external 
noise is parametrized by the effective noise amplitude p = p/y/N. 

Given the simplified description (2), the suppression of response fluctuations by negative 
feedback can be understood intuitively: Consider first the case where the neurons in the local 
network are unconnected (Fig. 3A; no feedback, w = 0). Here, the response r(t) (Fig. 3A 3 ) is 
simply a low-pass filtered version of the external input x(t) (Fig. 3Ai), resulting in an exponen- 
tially decaying response auto-correlation (Fig. 3D; light gray curve) and a drop in the response 
power-spectrum at the cutoff frequency f c (Fig. 3E). At low frequencies, r(t) and x(t) are 
in phase; they are correlated. In the presence of negative feedback (Fig. 3B), the local input 
—wr{t) (Fig. 3B 2 ) and the low-frequency components of the external input x(t) (Fig. 3Bi) 
are anticorrelated. They partly cancel out, thereby reducing the response fluctuations r(t) 
(Fig. 3B 3 ). The auto-correlation function and the power-spectrum are suppressed (Fig. 3D, E; 
black curves). Due to the low-pass characteristics of the system, mainly the low-frequency 
components of the external drive x(t) are transferred to the output side and, in turn, become 
available for the feedback signal. Therefore, the canceling of input fluctuations and the result- 
ing suppression of response fluctuations are most efficient at low frequencies. Consequently, 
the auto-correlation function is sharpened (see inset in Fig. 3D). The cutoff frequency of the 
system is increased (Fig. 3E; black curve). This effect of negative feedback is very general and 
well known in the engineering literature. It is employed in the design of technical devices, like, 
e.g., amplifiers [51]. As the zero- frequency power is identical to the integrated auto-correlation 
function, the suppression of low-frequency fluctuations is accompanied by a reduction in the 
auto-correlation area (Fig. 3D; black curve). Note that the suppression of fluctuations in the 
feedback case is not merely a result of the additional inhibitory noise source provided by the 
local input, but follows from the precise temporal alignment of the local and the external 
input. To illustrate this, let's consider the case where the feedback channel is replaced by a 
feedforward input q(t) (Fig. 3C) which has the same auto-statistics as the response r(t) in the 
feedback case (Fig. 3B 3 ) but is uncorrected to the external drive x(t). In this case, external 
input fluctuations (Fig.3Ci) are not canceled by the local input —wq(t) (Fig. 3C 2 ). Instead, 
the local feedforward input acts as an additional noise source which leads to an increase 
in the response fluctuations (Fig. 3C 3 ). The response auto-correlation and power-spectrum 
(Fig. 3D, E; dark gray curves) are increased. Compared to the unconnected case (Fig. 3E; light 
gray curve), the cutoff frequency remains unchanged. 
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Figure 3. Partial canceling of fluctuations in a linear system by inhibitory feedback. Response 
r{t) of a linear system with impulse response h(t) (lst-order low-pass, cutoff frequency 100 Hz) 
to Gaussian white noise input x(t) with amplitude p = 1 for three local-input scenarios. A 
(light gray): No feedback (local input q(t) = 0). B (black): Negative feedback (q(t) = 
r(t)) with strength w = 5. The fluctuations of the weighted local input —wq(t) (B 2 ) are 
anticorrelated to the external drive x(t) (Bi). C (dark gray): Feedback in B is replaced by 
uncorrected feedforward input q(t) with the same auto-statistics as the response r(t) in B 3 . 
The local input q(t) = ^J -1 [|i?(o;)|e 1 ^] (t) is constructed by assigning a random phase £(cj) 
to each Fourier component R(lj) = $[r(t)] (cj) of the response in B 3 . Fluctuations in C 2 and 
Ci are uncorrected. A,B,C: Network sketches. A^B^Ci: External input x{t). A2,B 2 ,C2: 
Weighted local input —wq(t). A3 7 B 3j C 3 : Responses r(t). D,E: Response auto-correlation 
functions (D) and power-spectra (E) for the three cases shown in A,B,C (same gray coding as 
in A,B,C; inset in D: normalized auto-correlations). 



The feedback induced suppression of response fluctuations can be quantified by comparing 
the response power-spectra 



and 



HP 



H(u)\\w'C RR (u)+p') 



(3) 



(4) 



in the feedback (Fig. 3B) and the feedforward case (Fig. 3C), respectively (see Sec. 4.5). Here, 
R(lj) and R(uj) denote the Fourier transforms of the response fluctuations in the feedback 
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and the feedforward scenario, respectively, H(uS) the transfer function (Fourier transform of 
the filter kernel h(t)) of the neuron population, and E x [] the average across noise realizations. 
We use the power ratio 

/ \ _ C rr (uj) _ 1 

a[U> ~ Ctt(u>)~vP\H{u)\* + \l + iDH(u,)\* U 

as a measure of the relative fluctuation suppression caused by feedback. For low frequencies 
(uj —> 0) and strong effective coupling \w\ = \Kw\ 3> 1, the power ratio (5) decays as w~ 2 
(see Fig. 4A): the suppression of population-rate fluctuations is promoted by strong negative 
feedback. In line with the observations in Sec. 2.1, this suppression is restricted to low fre- 
quencies; for high frequencies (uj — >■ oc, i.e. H(uj) — >• 0), the power ratio a(uj) approaches 1. 
Note that the power ratio (5) is independent of the amplitude p of the population averaged 
external input x(t). Therefore, even if we dropped the assumption of the external inputs xi(t) 
being uncorrected, i.e. if [xi(t)xj(t + r)] ^ for i ^ j, the power ratio (5) remained the 
same. For correlated external input, the power p of the population average x(t) is different 
from p/y/N. The suppression factor a(uj), however, is not affected by this. Moreover, it is 
straightforward to show that the power ratio (5) is, in fact, independent of the shape of the 
external-noise spectrum Cxx(w) = E x [|X(u;)| 2 ]. The same result (5) is obtained for any type 
of external input (e.g. colored noise or oscillating inputs). 




Figure 4. Suppression of low-frequency (LF) population-rate fluctuations in linearized homo- 
geneous random networks with purely inhibitory (A) and mixed excitatory-inhibitory coupling 
(B). Dependence of the zero-frequency power ratio a(0) on the effective coupling strength w 
(solid curves: full solutions; dashed lines: strong-coupling approximations). The power ratio 
a(0) represents the ratio between the low-frequency population-rate power in the recurrent 
networks (A: Fig. 3B; B: Fig. 5A,B) and in networks where the feedback channels are replaced 
by uncorrected feedforward input (A: Fig. 3C; B, black: Fig. 5C,D; B, gray: Fig. 5D f ). Dotted 
curves in B depict power ratio of the sum modes r + and f+ (see text). B: Balance factor 
2 = 1.5. 



For low frequencies, the transfer function H(uj) approaches unity (lim^ ^(^) = 1); 
the exact shape of the kernel h(t) becomes irrelevant. In particular, the cutoff frequency 
(or time constant) of a low-pass kernel has no effect on the zero- frequency power (integral 
correlation) and the zero- frequency power ratio a(0) (Fig. 4). Therefore, the suppression of 
low-frequency fluctuations does not critically depend on the exact choice of the neuron, synapse 
or input model. The same reasoning applies to synaptic delays: Replacing the kernel h(t) by 
a delayed kernel h(t — d) leads to an additional phase factor e~ lujd in the transfer function 
H(uS). For sufficiently small frequencies (long time scales), this factor can be neglected 
(lim^ e-^ = 1). 
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For networks with purely inhibitory feedback, the absolute power (3) of the population 
rate decreases monotonously with increasing coupling strength w. As we will demonstrate 
in Sec. 2.3 and Sec. 2.4, this is qualitatively different in networks with mixed excitatory and 
inhibitory coupling = w > and w\ = —gw < 0, respectively: here, the fluctuations of 
the compound activity increase with w. The power ratio a(oj), however, still decreases with 
w. 



2.3 Population-activity fluctuations in excitatory-inhibitory networks 

In the foregoing subsection, we have shown that negative feedback alone can efficiently suppress 
population-rate fluctuations and, hence, spike-train correlations. So far, it is unclear whether 
the same reasoning applies to networks with mixed excitatory and inhibitory coupling. To 
clarify this, we now consider a random network composed of a homogeneous excitatory and 
inhibitory subpopulation £ and X of size N E = \£\ and N\ = \I\ = jN E , respectively. Each 
neuron receives K = eN^ excitatory and = eN\ inhibitory inputs from £ and I with 
synaptic weights w > and — gw < 0, respectively. In addition, the neurons are driven by 
external Gaussian white noise. As demonstrated in Sec. 4.2, linearization and averaging across 
subpopulations leads to a two-dimensional system 

r(t) = ([Wr + x]*h)(t) (6) 

describing the linearized dynamics of the subpopulation averaged activity r(t) = (r E (i), ri(t)) T . 

Here, x(t) = (x E (t), x\(t)) T denotes the subpopulation averaged external uncorrected white- 

2; 



noise input with correlation functions E Xit [x p (t),x q (t + r)] = PpS pq S(r) (p p = pj y^, p,^G 



v 

{E, I}), and h(t) a normalized linear kernel with / °°dt h(t) = 1. The excitatory and inhibitory 
subpopulations are coupled through an effective connectivity matrix 

w-*Q :|) ( 7) 

with effective weight w = Kw > and balance parameter g = > 0. 

The two-dimensional system (6)/(7) represents a recurrent system with both positive and 
negative feedback connections (Fig. 5A). By introducing new coordinates 

r+(t) = (r E (t) + n(t))/V2, r_(t) = (r E (t) - r,(t))/^2 (8) 

and x + (t) = (xE.(t) + x\(t))/\/2, X-(t) = (^e(^) — x\(t))/\/2, we obtain an equivalent 
representation of (6)/(7), 



r+(t) 
r-(t) 



s[ r+ ) + ( x x + _ 



h\(t), (9) 



describing the dynamics of the sum and difference activity r+(t) and respectively, i.e. the 

in- and anti-phase components of the excitatory and inhibitory subpopulations (see [79, 83, 
47]). The new coupling matrix 

-W + Wff 



• < 10 > 

reveals that the sum mode r+(t) is subject to self-feedback (Sn = — w+ = w(l — g)) 
and receives feedforward input from the difference mode r_(t) (S\2 = w ff = ^(1 + d))- 
All remaining connections are absent (S 2 i = S 2 2 = 0) in the new representation (8) (see 
Fig. 5B). The correlation functions of the external noise in the new coordinates are given by 

E*,t [x p (t),x q (t + r)] = p 2 pq 5(r) with p pq = p 2 /N E ( 7 " 1 ^ + (1 - J~ 1 )/2) (p, q G {+, -}). 
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The feedforward coupling is positive (w F f > 0): an excitation surplus (r_(t) > 0) will 
excite all neurons in the network, an excitation deficit (r_(t) < 0) will lead to global in- 
hibition. In inhibition dominated regimes with g = > 1, the self-feedback of the sum 
activity r+(t) is effectively negative (— w+ < 0). The dynamics of the sum rate in inhibition- 
dominated excitatory-inhibitory networks is therefore qualitatively similar to the dynamics in 
purely inhibitory networks (Sec. 2.2). As shown below, the negative feedback loop exposed by 
the transform (8) leads to an efficient relative suppression of population-rate fluctuations (if 
compared to the feedforward case). 





W 




Re 








-gw 







B 



-gw 



X, 



Re 



X e 



-gw 



j-gw 

Q\ i 



R\ 

r 



Q- 



D' 



R. 



X- 







Wff 




R+ 


— w+ 



R- 



x 4 



R+ 



x. 



-w+ 



0^ 



Figure 5. Sketch of the 2D (excitatory-inhibitory) model for the feedback (A,B) and the 
feedforward scenario (C,D) in normal (A,C) and Schur-basis representation (B,D). A: Original 
2D recurrent system. B: Schur-basis representation of the system shown in A. C: Feedforward 
scenario: Excitatory and inhibitory feedback connections of the original network (A) are re- 
placed by feedforward input from populations with rates Qe, Q\, respectively. D: Schur-basis 
representation of the system shown in C. D': Alternative feedforward scenario: Here, the feed- 
forward channel (weight Wff) of the original system in Schur basis (B) remains intact. Only 
the inhibitory feedback (weight — w+) is replaced by feedforward input Q+. 



Mathematically, the coordinate transform (8) corresponds to a Schur decomposition of 
the dynamics: Any recurrent system of type (6) (with arbitrary coupling matrix W) can 
be transformed to a system with a triangular coupling matrix (see e.g. [47]). The resulting 
coupling between the different Schur modes can be ordered so that there are only connections 
from modes with lower index to modes with the same or larger index. In this sense, the 
resulting system has been termed 'feedforward' [47]. The original coupling matrix W is 
typically not normal, i.e. W T W ^ WW 7 . Its eigenvectors do not form an orthogonal 
basis. By performing a Gram-Schmidt orthonormalization of the eigenvectors, however, one 
can obtain a (normalized) orthogonal basis, a Schur basis. Our new coordinates (8) correspond 
to the amplitudes (the time evolution) of two orthogonal Schur modes. 

The spectra Cr^uj), C^^u)), C ReRx {uj) and C r+r+ (uj) of the subpopulation averaged 
rates r E , n and the sum mode r + , respectively, are derived in Sec. 4.6. In contrast to the purely 
inhibitory network (see Sec. 2.2), the population-rate fluctuations of the excitatory-inhibitory 
network increase monotonously with increasing coupling strength w. For strong coupling, 
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Cr + r + {uj) approaches 



w 



2 



Jim C R+R+ (u;) = |ff( w )|Y_^_-tt (11) 



from below with wpp/w+ = (g + l)/(g — 1). Close to the critical point (g ~ 1), the rate 
fluctuations become very large; (11) diverges. Increasing the amount of inhibition by increasing 
g, however, leads to a suppression of these fluctuations. In the limit g — >> oo, Cr + r + (lu) and 
(11) approach the spectrum lim^o C R+R+ = \H\ 2 p 2 (l + 7 -1 )/(2iV E ) of the unconnected 
network. For strong coupling (w 3> 1), the ratio C Re r e (uj) / C Rx r x {uj) approaches g 2 \ the 
fluctuations of the population averaged excitatory firing rate exceed those of the inhibitory 
population by a factor g 2 (independently of H(uS) and uj). 

Similarly to the strategy we followed in the previous subsections, we will now compare 
the population-rate fluctuations of the feedback system (6), or equivalently (9), to the case 
where the feedback channels are replaced by feedforward input with identical auto-statistics. 
A straight-forward implementation of this is illustrated in Fig. 5C: Here, the excitatory and 
inhibitory feedback channels Re and R\ are replaced by uncorrected feedforward inputs Qe 
and Q\, respectively. The Schur representation of this scenario is depicted in Fig. 5D. According 
to (6), the Fourier transforms of the response fluctuations of this system read 




= H(u) 




(12) 



With R+ = (R E + R\)/V%, and using C QeQe = C ReRe , C QxQx = C RxRv C QeQx = C QeXe = 
Cq e X\ = Cq^x e = CqiX, — 0, we can express the spectrum ^ (lj) of the sum activity in 
the feedforward case in terms of the spectra Cr e r e (uj) and Cr x r x {uj) of the feedback system 
(see eq. (55)). For strong coupling (w 3> 1), the zero- frequency component (H(0) = 1) 
becomes 

C k k (0) ~ w 2 p 2 l+ J~\ 4 ^\ 9 . (13) 
R + R + y Ne (g — l) 2 v ; 

Thus, for strong coupling, the zero- frequency power ratio 

reveals a relative suppression of the population-rate fluctuations in the feedback system which 
is proportional to 1/w 2 (see Fig. 4B; black dashed line). The power ratio a+(0) for arbitrary 
weights w is depicted in Fig. 4B (black dotted curve). For a network at the transition point 
g = 1, (14) equals l/(2w 2 ). Increasing the level of inhibition by increasing g leads to a 
decrease in the power ratio: in the limit g — >■ oc, (14) approaches l/(8w 2 ) monotonously. 

Above, we suggested that the negative self-feedback of the sum mode R +1 weighted 
by — w+ (Fig. 5B), is responsible for the fluctuation suppression in the recurrent excitatory- 
inhibitory system. Here, we test this by considering the case where this feedback loop is opened 
and replaced by uncorrected feedforward input Q +f weighted by — w+, while the feedforward 
input from the difference mode weighted by ^ff, is left intact (see Fig. 5D'). As before, 
we assume that the auto-statistics of Q + is identical to the auto-statistics of R+ as obtained 
in the feedback case, i.e. Cq + q + {uj) = C r+r+ {uj). According to the Schur representation of 
the population dynamics (9)/(10), the Fourier transform of the sum mode of this modified 
system is given by 

R + (u) = H(uj) (-w + Q + (uj) + w ff R_(cj) + X+(o;)) . (15) 
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With Cft + x + (oj) given in (54) and C r+r+ (uj), we obtain the power ratio 

f r \ _ Cr + r + (u) 1 , x 

a+[UJ) - C R+R+ (u)~ wl\H(uW + \l + w + H(u)\i- 1 ] 

Its zero- frequency component a+(0) is shown in Fig. 4B (gray dotted curve). For strong 
coupling, the power ratio decays as l/(2w+) (gray dashed line in Fig. 4B). Thus, the (relative) 
power in the recurrent system is reduced by strengthening the negative self-feedback loop, 
i.e. by increasing w+. 

So far, we have presented results for the subpopulation averaged firing rates r^(t) and 
r\(t) and the sum mode r+(t). The spectrum of the compound rate r(t) = N~ x Y^i=i r *(*) = 
7V -1 [7V E rE(£) + N\r\(t)] t i.e. the activity averaged across the entire population, reads 

Crr(u>) = N- 2 (NIC ReRe (uj) + N?C R[R[ (uj) + N e Ni[C ReR[ (uj) + CW")1) ■ ( 17 ) 

In the feedforward scenario depicted in Fig. 5C, the spectrum of the compound rate R = 

H(wQe + wgQ\ + X) (with X = N~ l Y^ = i X i) is S' ven b Y 

C M (u) = \H(u)\ 2 {w 2 C ReRe + w 2 fC RlRl + p 2 /N) . (18) 

For strong coupling, the corresponding low-frequency power ratio a(0) = C RR (0)/Cp>p>(0) 
(black solid curve in Fig. 4B) exhibits qualitatively the same decrease oc w~ 2 as the sum 
mode. 

To summarize the results of this subsection: the population dynamics of a recurrent net- 
work with mixed excitatory and inhibitory coupling can be mapped to a two-dimensional system 
describing the dynamics of the sum and the difference of the excitatory and inhibitory sub- 
population activities. This equivalent representation uncovers that, in inhibition dominated 
networks (g > 1), the sum activity is subject to negative self-feedback. Thus, the dynamics 
of the sum activity in excitatory-inhibitory networks is qualitatively similar to the popula- 
tion dynamics of purely inhibitory networks (see Sec. 2.2). Indeed, the comparison of the 
compound power-spectra of the intact recurrent network and networks where the feedback 
channels are replaced by feedforward input reveals that the (effective) negative feedback in 
excitatory-inhibitory networks leads to an efficient suppression of population-rate fluctuations. 



2.4 Population averaged correlations in cortical networks 

The results presented in the previous subsections describe the fluctuations of the compound 
activity. Pairwise correlations Qj(t) = E t t [si(t + t')sj(t')] between the (centralized) spike 
trains Si(t) = Si(t) — E t f [ s i(t')] are outside the scope of such a description. In this subsec- 
tion, we consider the same excitatory-inhibitory network as in Sec. 2.3 and present a theory 
for the population averaged spike-train cross-correlations. In general, this is a hard problem. 
To understand the structure of cross-correlations, it is however sufficient to derive a rela- 
tionship between the cross- and auto-covariances in the network, because the latter can, to 
good approximation, be understood in mean-field theory. The integral of the auto-covariance 
function of spiking LIF neurons can be calculated by Fokker-Planck formalism [11, 9, 46]. To 
determine the relation between the cross-covariance and the auto-covariance, we replace the 
spiking dynamics by a reduced linear model whose covariances, to linear order, obey the same 
relation. We present the full derivation in Sec. 4.2. There, we first derive an approximate 
linear relation between the auto- and cross-covariance functions a(r) and c(r), respectively, 
of the LIF network. A direct solution of this equation is difficult. In the second step, we 
therefore show that there exists a linear stochastic system with activity u whose correlations 
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a u (r) and c u (r) fulfill the same equation as the original LIF model. This reduced model can 
be solved in the frequency domain by standard Fourier methods. Its solution allows us, by 
construction, to determine the relation between the integral cross-covariances C(0) and the 
integral auto-covariances -4.(0) up to linear order. 

As we are interested in the covariances averaged over many pairs of neurons, we aver- 
age the resulting set of TV linear self-consistency equations (56) for the covariance matrix in 
the frequency domain C{u) over statistically identical pairs of neurons and many realizations 
of the random connectivity (see Sec. 4.7). This yields a four-dimensional linear system (76) 
describing the population averaged variances and A\ of the excitatory and inhibitory sub- 
populations, and the covariances Ce^e and C\^±\ for unconnected excitatory-excitatory and 
inhibitory-inhibitory neuron pairs, respectively (note that we use the terms "variance" and 
"covariance" to describe the integral of the auto- and cross-correlation function, respectively; 
in many other studies, they refer to the zero-lag correlation functions instead). The depen- 
dence of the variances and covariances on the coupling strength w, obtained by numerically 
solving (76), is shown in Fig. 6. We observe that the variances A E and A\ of excitatory and 
inhibitory neurons are barely distinguishable (Fig. 6A). With the approximation A := A E = A\, 
explicit expressions can be obtained for the covariances (thick dashed curves Fig. 6E): 

Cee/ " = (i - wl - w ^ ared (19) 

v v ' 

I 

1 . [4r for EE 



E 



+ ; r 2wA , _ 

l-w(l-g) 1^ for II 



ii 



in 



Cei — 2 ^ EE ~^ 
with Ci« d = &(±- + Q A . 

The deviations from the full solutions (thin solid curves in Fig. 6E), i.e. for A E ^ A\, are 
small. In the reduced model, both the external input and the spiking of individual neurons 
contribute to an effective noise. As the fluctuations in the reduced model depend linearly on 
the amplitude p of this noise, the variances A and covariances C pq (p,q G {E, I}) can be 
expressed in units of the noise variance p 2 . Consequently, the correlation coefficients C pq /A 
are independent of p 2 (see Fig. 6). 

The analytical form (19) of the result shows that the correlations are smaller than expected 
given the amount of shared input a pair of neurons receives: The quantity Qh ared in the first 
line is the contribution of shared input to the covariance. For strong coupling w ;$> 1, the 
prefactor I causes a suppression of this contribution. Its structure is typical for a feedback 
system, similar to the solution (3) of the one-population or the solution (52) of the two- 
population model. The term w(l — g) in the denominator represents the negative feedback of 
the compound rate. The prefactor II in the second line of (19) is again due to the feedback and 
suppresses the contribution of the factor III, which represents the effect of direct connections 
between neurons. 

Our results are consistent with a previous study of the decorrelation mechanism: In [57], 
the authors considered how correlations scale with the size TV of the network where the synaptic 
weights are chosen as J oc 1/y/N. As a result, the covariance Qh ared in (19) caused by shared 
input is independent of the network size, while the feedback w{l — g) oc eN(l — g) (J + 0{ J 2 )) 
scales — to leading order — as y/~N (see (45)). Consequently, the first line in (19) scales as 1/N. 
The same scaling holds for the second line in (19), explaining the decay of correlations as 1/N 
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found in [57]. 

The first line in (19) is identical for any pair of neurons. The second line is positive for a pair 
of excitatory neurons and negative for a pair of inhibitory neurons. In other words, excitatory 
neurons are more correlated than inhibitory ones. Together with the third line in (19), this re- 
veals a peculiar correlation structure: Cee > Cei > Cm (Fig.6B,E). For strong coupling w 3> 1, 
the difference between the excitatory and inhibitory covariance is Cee — Q\ — + tv,M- 

The difference decreases as the level g of inhibition is increased, i.e. the further the network 
is in the inhibition dominated regime, away from the critical point g = 1. 

To understand the suppression of shared-input correlations in recurrent excitatory-inhibitory 
networks, consider the correlation between the local inputs I k /i = [Wr]k/i of a pair of neu- 
rons k, I. The input-correlation coefficient C m /A m = Cov [I k) Ii] / ^/Var [I k ] Var [//] can be 
expressed in terms of the averaged spike-train covariances: 

C in = Cov [hJi] = C'- ared + C'Z r 

A- = Var [I k ] = e^w" {^ + Q A + <?«r (20) 
with C'Zr = ™ 2 (Cee ~ 2gC B + fC u ) 

(see Sec. 4.7: The input covariance C m equals the average quantity C xy ^ given in (67), the 
input variance A m is given by (63) as A x ^)- The term Qh ared represents the contribution due 
to the spike-train variances of the shared presynaptic neurons (see (19)). This contribution is 
always positive (provided the network architecture is consistent with Dale's law; see [37]). In 
a purely feedforward scenario with uncorrected presynaptic sources, C^ ared is the only con- 
tribution to the input covariance of postsynaptic neurons. The resulting response correlation 
for this feedforward case is much larger than in the feedback system (Fig. 6B, black dotted 
curve). The correlation coefficient between inputs to a pair of neurons in the feedforward case 
is identical to the network connectivity e (horizontal dotted curve in Fig. 6D; see [37]). In 
an inhibition dominated recurrent network, spike-train correlations between pairs of different 
source neurons contribute the additional term C^ rr , which is negative and of similar absolute 
value as the shared-input contribution C^ ared . Thus, the two terms Cg[J ared and C™ orr partly 
cancel each other (see Fig. 6C). In consequence, the resulting input correlation coefficient 
C' m /A m is smaller than e (see Fig. 6D; here: e = 0.1). 

The correlations in a purely inhibitory network can be obtained from (19) by replacing 
Ne —> N, taking into account the negative sign of w in w = — Kw and setting g = and 
7 = 0: 

c ^(- 1 + (TT^)^' < 21 > 

For finite coupling strength w > 0, this expression is negative. The contributions of shared 
input and spike-train correlations to the input correlation are given by Qh ared = w 2 ^ > 
and C^ orr = w 2 C, respectively (see (19) and (20)). Using (21), we can directly verify that 
Qorr < 0' because pairwise correlations C are negative, leading to a partial cancellation 
Qhared + Qorr = ™ 2 (i+w)* W : t ^ ie r 'S^ hand side is smaller in magnitude by a factor of 
^ compared to each individual contribution. Hence, as in the network with excitation and 
inhibition, shared-input correlations are partly canceled by the contribution due to presynaptic 
pairwise spike-train correlations. In the feedforward scenario with zero presynaptic spike-train 
correlations, in contrast, the response correlations are determined by shared input alone and are 
therefore increased. The suppression of shared-input correlations in the feedback case is what 
we call 'decorrelation' in the current work. In purely inhibitory networks, this decorrelation is 
caused by weakly negative pairwise correlations (21). For sufficiently strong negative feedback, 
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correlations are smaller in absolute value as compared to the feedforward case. The absolute 
value of these anti-correlations is bounded by A/N. 

The similarity in the results obtained for purely inhibitory networks and excitatory-inhibitory 
networks demonstrates that the suppression of pairwise correlations and population-activity 
fluctuations is a generic phenomenon in systems with negative feedback. It does not rely on 
an internal balance between excitation and inhibition. 

As discussed in Sec. 2.1, the suppression of correlations in the recurrent network is ac- 
companied by a reduction of population-activity fluctuations. With the population averaged 
correlations (19), the power (1) of the population activity r(t) reads 

Crr = ^ [N E A E + AU, + N E (N E - 1)C EE + A^iV, - 1)C„ + 27V E 7V,C EI ] . (22) 

In Sec. 2.3, we showed that the population-activity fluctuations are amplified if the local input 
in the recurrent system is replaced by feedforward input from independent excitatory and 
inhibitory populations (see Fig. 5C). This manipulation corresponds to a neglect of correlations 
C E | between excitatory and inhibitory neurons. All remaining correlations (A E , A\, C EE , C\\) 
are preserved. With the resulting response auto- and cross-correlations A and C given by (84), 
the power (1) of the population activity becomes 

C™ = ii+(l-i)(5. (23) 

For large effective coupling w, the power ratio a = Crr/C^ decays as 1/w 2 (black curve 
in Fig. 6F). Note that the power ratio a derived here is indistinguishable from the one we 
obtained in the framework of the population model in Sec. 2.3 (black solid curve in Fig.4B). 
Although the derivation of the macroscopic model in Sec. 2.3 is qualitatively different from 
the one leading to the population averaged correlations described here, the two models are 
consistent: They describe one and the same system and lead to identical power ratios. 

The fluctuation suppression is not only observed at the level of the entire network, i.e. for 
the population activity r(t), but also for each individual subpopulation £ and X, i.e. for the 
subpopulation averaged activities r E (t) and r\(t). The derivation of the corresponding power 
ratios a E and ct\ is analog to the one described above. As a result of the correlation structure 
C EE > C\\ in the feedback system (see Fig. 6B), the power of the inhibitory population activity 
is smaller than the power of the excitatory population activity. In consequence, a E > ol\ (gray 
curves in Fig. 6F). 

In (22) and (23), the auto-correlations are scaled by 1/iV, while the cross-correlations enter 
with a prefactor of order unity. For large N, one may therefore expect that the suppression 
of population-activity fluctuations is essentially mediated by pairwise correlations. In the 
recurrent system, however, the cross-correlations C xy (x,y e {E, I}) are of order A/N (see 
Fig. 6 and (19)). It is therefore a priori not clear whether the fluctuation suppression is 
indeed dominated by pairwise correlations. In our framework, one can explicitly show that 
the auto-correlation is irrelevant: Replacing the auto-correlation A in (23) by the average 
auto-correlation (N E A E + N\A\)/N of the intact feedback system has no visible effect on the 
resulting power ratio (dashed curves in Fig. 6F). The difference in the spectra of the population 
activities Crr and C^ is therefore essentially caused by the cross-correlations. 

The absolute population-activity fluctuations in purely inhibitory and in excitatory-inhibitory 
networks show a qualitatively different dependence on the synaptic coupling w, in agreement 
with the previous sections. In networks with excitation and inhibition, the correlation coeffi- 
cient increases with increasing synaptic coupling (see Fig. 6E). Hence, the population-activity 
fluctuations grow with increasing coupling strength. In purely inhibitory networks, in contrast, 
the pairwise spike-train correlation decreases monotonously with increasing magnitude of the 
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coupling strength w, see (21). In consequence, the population-activity fluctuations decrease. 
The underlying reason is that, in the inhibitory network, the power of the population activity 
is directly proportional to the covariance of the input currents, which is actively suppressed, 
as shown above. For excitatory-inhibitory networks, these two quantities are not proportional 
(compare (20) and (1)) due to the different synaptic weights appearing in the input covariance. 

To compare our theory to simulations of spiking LIF networks, we need to determine the 
effect of a synaptic input on the response activity of the neuron model. To this end, we 
employ the Fokker-Planck theory of the LIF model (see Sec. 4.3). In this context, the steady 
state of the recurrent network is characterized by the mean /i and the standard deviation 
a of the total synaptic input. Both \i and a depend on the steady-state firing rate in the 
network. The steady-state firing rate can be determined in a self-consistent manner [9] as the 
fixed point of the firing rate approximation (42). The approximation predicts the firing rate 
to sufficient accuracy of about ±ls _1 (see Fig. 7A). We then obtain an analytical expression 
of the low-frequency transfer which relates the fluctuation Vj{t) = v + eS(t) of a synaptic 
input to neuron i to the fluctuation of neuron z's response firing rate to linear order, so that 
J °° 8vi(t) dt = ew(Jij). This relates the postsynaptic potential in the LIF model to the 
effective linear coupling = w(Jij) in our linear theory. The functional relation w(J) can be 
derived in analytical form by linearization of (42) about the steady-state working point. Note 
that w(J) depends on /i and a and, hence, on the steady-state firing rate in the network. The 
derivation outlined in Sec. 4.3 constitutes an extension of earlier work [16, 28] to quadratic 
order in J. The results agree well with those obtained by direct simulation for a large range 
of synaptic amplitudes (see Fig. 10). 

Fig. 7B compares the population averaged correlation coefficients C/A obtained from the 
linear reduced model, see (19), and simulations of LIF networks. Note that the absolute value 
of the noise amplitude p in the reduced model does not influence the correlation coefficient 
C/A, as both quantities C and A depend linearly on p 2 . Theory and simulation agree well for 
synaptic weights up to J ^ 1 mV. For larger synaptic amplitudes, the approximation of the 
effective linear transfer for a single neuron obtained from the Fokker-Planck theory deviates 
from its actual value (see Fig. 10B). Fig. 7C shows that the cancellation of the input covariance 
in the LIF network is well explained by the theory. 

Previous work [57] suggested that positive correlations between excitatory and inhibitory 
inputs lead to a negative component in the input correlation which, in turn, suppresses shared- 
input correlations. The mere existence of positive correlations between excitatory and inhibitory 
inputs is however not sufficient. To explain the effect, it is necessary to take the particular cor- 
relation structure Cee > Cei > Q\ into account. To illustrate this, consider the case where the 
correlation structure is destroyed by replacing all pairwise correlations in the input spike-train 
ensemble by the overall population average C = (A^Cee + N\C\\)/(N E + N\) > (homoge- 
nization of correlations). The resulting response correlations (upper gray curve in Fig. 7B) are 
derived in Sec. 4.7, eq. (86). In simulations of LIF networks, we study the effect of homogenized 
spike-train correlations by first recording the activity of the intact recurrent network, randomly 
reassigning the neuron type (E or I) to each recorded spike train, and feeding this activity 
into a second population of neurons. Compared to the intact recurrent network, the response 
correlations are significantly larger (Fig. 7B). The contribution of homogenized spike-train cor- 
relations to the input covariance C m (see (20)) is given by C^ orr hom = w 2 (l — g) 2 C > 0. For 
positive spike-train correlations C > 0, this contribution is greater or equal zero (zero for 
g = 1). Hence, it cannot compensate the (positive) shared-input contribution Qh ared (see 
Fig. 7C). In consequence, input correlations, output correlations and, in turn, population-rate 
fluctuations (Fig. 7D) cannot be suppressed by homogeneous positive correlations in the input 
spike-train ensemble. Canceling of shared-input correlations requires either negative spike-train 
correlations (as in purely inhibitory networks) or a heterogeneity in correlations across different 
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pairs of neurons (e.g. Cee > Cei > Cm)- 

2.5 Effect of feedback manipulations 

In the previous subsections, we quantified the suppression of population-rate fluctuations in 
recurrent networks by comparing the activity in the intact recurrent system (feedback scenario) 
to the case where the feedback is replaced by feedforward input with some predefined statistics 
(feedforward scenario). We particularly studied the effect of neglecting the auto-statistics of 
the compound feedback, (the structure of) correlations within the feedback ensemble and/or 
correlations between the feedback and the external input. In all cases, we observed a significant 
amplification of population-activity fluctuations in the feedforward scenario. In this subsection, 
we further investigate the role of different types of feedback manipulations by means of simu- 
lations of LIF networks with excitatory-inhibitory coupling. To this end, we record the spiking 
activity of the recurrent network (feedback case), apply different types of manipulations to this 
activity (described in detail below) and feed this modified activity into a second population 
of identical (unconnected) neurons (feedforward case). As before, the connectivity structure 
(in-degrees, shared-input structure, synaptic weights) is exactly identical in the feedback and 
the feedforward case. 

In Sec. 4.2, we show that the low-frequency fluctuations of the population rate s(t) of the 
spiking model are captured by the reduced model r(t) presented in the previous subsections. 
To verify that the theory based on excitatory and inhibitory population rates is indeed sufficient 
to explain the decorrelation mechanism, we first consider the case where the sender identities 
of the presynaptic spike train are randomly shuffled. Fig. 8A shows the power-spectrum of the 
population activity recorded in the original network (FB) as well as the spectra obtained after 
shuffling spike-train identities within the excitatory and inhibitory subpopulations separately 
(Shuff2D), or across the entire network (ShufflD). As shuffling of neuron identities does not 
change the population rates, all three compound spectra are identical. Fig. 8B shows the 
response power-spectra of the neuron population receiving the shuffled spike trains. Shuf- 
fling within the subpopulations (Shuff2D) preserves the population-specific fluctuations and 
average correlations. The effect on the response fluctuations is negligible (compare black and 
light gray curves in Fig. 8B). In particular, the power of low-frequency fluctuations remains 
unchanged (Fig. 8C). This result confirms that population models which take excitatory and 
inhibitory activity separately into account are sufficient to explain the observations. Shuffling 
of spike-train identities across subpopulations (ShufflD), in contrast, causes an increase in the 
population fluctuations by about one order of magnitude (Fig. 8B,C; dark gray). This outcome 
is in agreement with the result obtained by homogenizing pairwise correlations (see Fig. 7) and 
demonstrates that the excitatory and inhibitory subpopulation rates have to be conserved to 
explain the observed fluctuation suppression. 

The shuffling experiments and the results of the linear model in the previous subsec- 
tions suggest that the precise temporal structure of the population averaged activities within 
homogeneous subpopulations is essential for the suppression of population-rate fluctuations. 
Preserving the exact structure of individual spike trains is not required. This is confirmed by 
simulation experiments where new sender identities were randomly reassigned for each indi- 
vidual presynaptic spike (rather than for each spike train; data not shown). This operation 
destroys the structure of individual spike trains but preserves the compound activities. The 
results are similar to those reported here. 

So far, it is unclear how sensitive the fluctuation-suppression mechanism is to perturbations 
of the temporal structure of the population rates. To address this question, we replaced the 
excitatory and inhibitory spike trains in the feedback ensemble by independent realizations of 
inhomogeneous Poisson processes (Poissl) with intensities given by the measured excitatory 
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and inhibitory population rates s E (t) and s\(t) of the recurrent network, respectively. Note 
that the compound rates of a single realization of this new spike-train ensemble are similar but 
not identical to the original population rates s E (t), s\(t) (in each time window [t + At), the 
resulting spike count is a random number drawn from a Poisson distribution with mean and 
variance proportional to se(^) and s\(t), respectively). Although the compound spectrum of 
the resulting local input is barely distinguishable from the compound spectrum of the intact 
recurrent system (Fig. 8D; black and dark gray curves), the response spectra are very different: 
replacing the feedback ensemble by inhomogeneous Poisson processes leads to a substantial 
amplification of low-frequency fluctuations (Fig. 8E; compare black and dark gray curves). 
The effect is as strong as if the temporal structure of the population rates was completely 
ignored, i.e. if the feedback channels were replaced by realizations of homogeneous Poisson 
processes with constant rates (PoissH; light gray curves in Fig. 8D,E). This result indicates 
that the precise temporal structure of the population rates is essential and that even small 
deviations can significantly weaken the fluctuation-suppression mechanism. The results of the 
Poisson experiments can be understood by considering the effect of the additional noise caused 
by the stochastic realization of individual spikes. Considering the auto-correlation, a Poisson 
spike-train ensemble with rate profile is(t) is equivalent to a sum of the rate profile and a noise 
term resulting from the stochastic (Poissonian) realization of spikes, q(t) = v(t)l + sjv^zit). 
Here, z(t) denotes a Gaussian white noise with auto-correlation E t [z(t)z(t + s)] = 15 (s) and 
is = E t [v(t)\ the mean firing rate. The response fluctuations of the population driven by the 
rate modulated Poisson activity are, to linear approximation, given by R = H{WQ + X). 
Inserting Q, we obtain an additional noise term v \H\ 2 WW T in the spectrum Cp>p> = RR* 
which explains the increase in power compared to the spectrum Crr of the recurrent network. 
As a generalization of the Poisson model, one may replace the noise amplitude sjv^ by some 
arbitrary prefactor 77. In simulation experiments, we observed a gradual amplification of the 
population-rate fluctuations with increasing noise amplitude rj (data not shown). 

3 Discussion 

We have shown that negative feedback in recurrent neural networks actively suppresses low- 
frequency fluctuations of the population activity and pairwise correlations. This mechanism 
allows neurons to fire more independently than expected given the amount of shared presy- 
naptic input. We demonstrated that manipulations of the feedback statistics, e.g. replacing 
feedback by uncorrected feedforward input, can lead to a significant amplification of response 
correlations and population-rate fluctuations. 

The suppression of correlations and population-rate fluctuations by feedback can be ob- 
served in networks with both purely inhibitory and mixed excitatory-inhibitory coupling. In 
purely inhibitory networks, the effect can be understood by studying the role of the effec- 
tive negative feedback experienced by the compound activity. In networks of excitatory and 
inhibitory neurons, a change of coordinates, technically a Schur decomposition, exposes the 
underlying feedback structure: the sum of the excitatory and inhibitory activity couples neg- 
atively to itself if the network is in an inhibition dominated regime (which is required for its 
stability; see e.g. [9]). This negative feedback suppresses fluctuations in a similar way as in 
purely inhibitory networks. The fluctuation suppression becomes more efficient the further 
the network is brought into the inhibition dominated regime, away from the critical point of 
equal recurrent excitation and inhibition (g = 1). Having identified negative feedback as the 
underlying cause of small fluctuations and correlations, we can rule out previous explanations 
based on a balance between (correlated) excitation and inhibition [57]. We presented a self- 
consistent theory for the average pairwise spike-train correlations which illuminates that the 
suppression of population-rate fluctuations and the suppression of pairwise correlations are 
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two expressions of the same effect: as the single spike-train auto-covariance is the same in 
the feedforward and the feedback case, the suppression of population-rate fluctuations implies 
smaller correlations. Our theory enables us to identify the cancellation of input correlations as 
a hallmark of small spike-train correlations. 

In previous studies, shared presynaptic input has often been considered a main source of 
correlation in recurrent networks (e.g. [64, 37]). Recently, [57] suspected that correlations 
between excitatory and inhibitory neurons and the fast tracking of external input by the ex- 
citatory and the inhibitory population are responsible for an active decorrelation. We have 
demonstrated here that the mere fact that excitatory and inhibitory neurons are correlated 
is not sufficient to suppress shared-input correlations. Rather, we find that the spike-train 
correlation structure in networks of excitatory and inhibitory networks arranges such that their 
overall contribution to the covariance between the summed inputs to a pair of neurons becomes 
negative, canceling partly the effect of shared inputs. This cancellation becomes more precise 
the stronger the negative compound feedback Kw(l — ^g) is. In homogeneous networks where 
excitatory and inhibitory neurons receive statistically identical input, the particular structure 
of correlations is Cee > Cei > C\\. It can further be shown that this structure of correlations 
is preserved in the limit of large networks TV — >> oc (K/N = const.). For non-homogeneous 
synaptic connectivity, if the synaptic amplitudes depend on the type of the target neuron (i.e. 
Jee 7^ J\e or Jei 7^ ^n), the structure of correlations may be different. Still, the correlation 
structure arranges such that shared input correlation is effectively suppressed. Formally, this 
can be seen from a self-consistency equation similar to our equation (80). 

The study by [57] has shown that correlations are suppressed in the limit of infinitely 
large networks of binary neurons receiving randomly drawn inputs from a common external 
population. Its argument rests on the insight that the population-activity fluctuations in a 
recurrent balanced network follow the fluctuations of the external common population. An 
elegant scaling consideration for infinitely large networks TV — » oc with vanishing synaptic 
efficacy oc 1/y/N shows that this fast tracking becomes perfect in the limit. This allows 
to determine the zero-lag pairwise correlations caused by the external input. The analysis 
methods and the recurrent networks presented here differ in several respects from these previous 
results: We study networks of a finite number of spiking model neurons. The neurons receive 
uncorrected external input, so that correlations are due to the local recurrent connectivity 
among neurons, not due to tracking of the common external input [57]. Moreover, we consider 
homogeneous connectivity where synaptic weights depend only on the type of the presynaptic 
neuron (as, e.g., in [9]), resulting in a correlation structure Cee > Cei > Cii- For such 
connectivity, networks of binary neurons with uncorrected external input exhibit qualitatively 
the same correlation structure as reported here (results not shown). 

In purely inhibitory networks, the decorrelation occurs in an analog manner as in excitatory- 
inhibitory networks. As only a single population of neurons is available here, population 
averaged spike-train correlations C\\ are negative. This negative contribution compensates the 
positive contribution of shared input. 

The structure of integrated spike-train covariances in networks constitutes an experimen- 
tally testable prediction. Note, however, that the prediction (19) obtained in the current work 
rests on two simplifying assumptions: identical internal dynamics of excitatory and inhibitory 
neurons and homogeneous connectivity (i.e. Jee = Jie, Je\ = J\\\ see Sec. 2.3). For such 
networks, the structure of correlations is given by Cee > Cei > C\\. Further, the relation 
between subthreshold membrane-potential fluctuations and spike responses is the same for 
both neuron types. Consequently, the above correlation structure can be observed not only at 
the level of spike trains but also for membrane potentials, provided the assumptions hold true. 
A recent experimental study [21] reports neuron-type specific cross-correlation functions in the 
barrel cortex of behaving mice, both for spike trains and membrane potentials. It is however 
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difficult to assess the integral correlations from the published data. A direct test of our pre- 
dictions requires either a reanalysis of the data or a theory predicting the entire correlation 
functions. The raw (unnormalized) II and El spike-train correlations in [21] are much more 
pronounced than the EE correlations (Fig. 6 in [21]). This seems to be in contradiction to 
our results. Note, however, that the firing rates of excitatory and inhibitory neurons are very 
different in [21]. In our study, in contrast, the average firing rates of excitatory and inhibitory 
neurons are identical as a consequence of the assumed network homogeneity. Future theoret- 
ical work is needed to generalize our model to networks with heterogeneous firing rates and 
non-homogeneous connectivity. Recent results on the dependence of the correlation structure 
on the connectivity may prove useful in this endeavor [52, 53, 78]. 

Correlations in spike-train ensembles play a crucial role for the en- and decoding of in- 
formation. A set of uncorrected spike trains provides a rich dynamical basis which allows 
readout neurons to generate a variety of responses by tuning the strength and filter properties 
of their synapses [77]. In the presence of correlations, the number of possible readout signals 
is limited. Moreover, spike-train correlations impair the precision of such readout signals in 
the presence of noise. Consider, for example, a linear combination y(t) = YliLi( s i * hj)(t) 
of TV presynaptic spike trains with arbitrary (linear) filter kernels hi(t) (e.g. synaptic filters). 
In a realistic scenario, the individual spike trains si(t) typically vary across trials [67, 63]. To 
understand how robust the resulting readout signal y(t) is against this spike-train variability, 
let's consider the variability of its Fourier transform Y(lj) = $[y(t)] (lj) = J2iLi Si(u)Hi(uj). 
Assuming homogeneous spike-train statistics, 

S(u) := E; [Si(u>)] (mean) 

V(oj) := E; [\Si(u) - S{u)\ 2 ] (variance) (24) 

C(u) := E^j [(Si(u>) - S(u)) (Sj(u) - S(cj))*] (covariance) , 

the (squared) signal-to-noise ratio of the readout signal Y{uS) is given by 

c m2 / . . = |E[^)]| 2 = I£| 2 |^l| 2 (2 n 

E [\Y{u) - E \Y(u)] | 2 ] N-^l - k)VH 2 + kV\H x \* ' K ] 

Here, k{uj) = C(cj)/V(lj) denotes the spike-train coherence. The coefficients Hi := E^ [Hi] 
and H 2 := E^ [\Hi\ 2 ] represent the 1st- and 2nd-order filter statistics. For uncorrected spike 
trains, i.e. k(uj) = 0, and S(uS) ^ 0, the signal-to-noise ratio SN 2 grows unbounded with the 
population size N. Thus, even for noisy spike trains (V > 0), the compound signal y(t) can 
be highly reliable if the population size TV is sufficiently large. In the presence of correlations, 
K,(cj) 0, however, SN 2 converges towards a constant value ft" 1 !^ 2 !^ -1 as TV grows. Even 
for large populations, the readout signal remains prone to noise. These findings constitute a 
generalization of the results reported for population-rate coding, i.e. sums of unweighted spike 
counts (see e.g. [85, 63]). The above arguments illustrate that the same reasoning applies to 
coding schemes which are based on the spatio-temporal structure of spike patterns. 

In a previous study [73], we demonstrated that active decorrelation in recurrent networks is a 
necessary prerequisite for a controlled propagation of synchronous volleys of spikes in embedded 
feedforward subnetworks ('synfire chains'; Fig. 9): A synfire chain receiving background input 
from a finite population of independent Poisson sources amplifies the resulting shared-input 
correlations, thereby leading to spontaneous synchronization within the chain (Fig. 9B). A 
distinction between these spurious synchronous events and those triggered by an external 
stimulus is impossible. The synfire chain loses its asynchronous ground state [72]. A synfire 
chain receiving background inputs from a recurrent network, in contrast, is much more robust. 
Here, shared-input correlations are actively suppressed by the recurrent-network dynamics. 
Synchronous events can be triggered by external stimuli in a controlled manner (Fig. 9A). 
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Apart from the spontaneous synchronization illustrated in Fig. 9, decorrelation by inhibition 
might solve another problem arising in embedded synfire structures: In the presence of feedback 
connections between the synfire chain and the embedding background network, synchronous 
spike volleys can excite (high-frequency) oscillatory modes in the background network which, 
in turn, interfere with the synfire dynamics and prevent a robust propagation of synchronous 
activity within the chain ('synfire explosion', see [42, 3]). The decorrelation mechanism we 
refer to in our work is efficient only at low frequencies. It cannot prevent the build-up of 
these oscillations. [2] demonstrated that the 'synfire explosion' can be suppressed by adding 
inhibitory neurons to each synfire layer ('shadow inhibition') which diffusely project to neurons 
in the embedding network, thereby weakening the impact of synfire activity on the embedding 
network. 

In the present work we focus on the integral of the correlation function, nurtured by 
our interest in the low-frequency fluctuations. An analog treatment can however easily be 
performed for the zero-lag correlations. In contrast to infinite networks with sparse connectivity 
(N ^ oo, K = const), in the case of finite networks, pairs of neurons must be distinguished 
according to whether they are synaptically connected or not in order to arrive at a self- 
consistent theory for the averaged correlations. Providing explicit expressions for correlations 
between connected and unconnected neurons, the current work provides the tools to relate 
experimentally observed spiking correlations to the underlying synaptic connectivity. 

The quantification of pairwise correlations is a necessary prerequisite to understand how 
correlation sensitive synaptic plasticity rules, like spike-timing dependent plasticity [5], interact 
with the recurrent network dynamics [23]. Existing theories quantifying correlations employ 
stochastic neuron models and are limited to purely excitatory networks [13, 23, 54]. Here, we 
provide an analytical equivalence relation between a reduced linear model and spiking integrate- 
and-fire neurons describing fluctuations correctly up to linear order. A formally similar approach 
has been employed earlier to study delayed cumulative inhibition in spiking networks [39]. We 
show that the correlations observed in recurrent networks in the asynchronous irregular regime 
are quantitatively captured for realistic synaptic coupling with postsynaptic potentials of up to 
about 1 mV. The success of this approach can be explained by the linearization of the neural 
threshold units by the afferent noise experienced in the asynchronous regime. For linear neural 
dynamics, the second-order description of fluctuations is closed [12]. We exploit this finding by 
applying perturbation theory to the Fokker-Planck description of the integrate-and-fire neuron 
to obtain the linear input-output transfer at low frequencies [28], thereby determining the 
effective coupling in our linear model. 

The scope of the theory presented in the current work is limited mainly by three assump- 
tions. The first is the use of a linear theory which exhibits an instability as soon as a single 
eigenvalue of the effective connectivity matrix assumes a positive real part. This ultimately 
happens when increasing the synaptic coupling strength, because the eigenvalues of the ran- 
dom connectivity matrix are located in a circle centered in the left half of the complex plain 
with a radius given by the square root of the variance of the matrix elements [68, 56]. Non- 
linearities, like those imposed by strictly positive firing rates, prevent such unbounded growth 
(or decay) by saturation. For nonlinear rate models with sigmoidal transfer functions it has 
been shown that the activity of recurrent random networks of such units makes a transition 
to chaos at the point where the linearized dynamics would loose stability [69]. However, this 
point of transition is sharp only in the limit of infinitely large networks. From the population 
averaged firing rate and the pairwise correlations averaged over pairs of neurons considered in 
Fig. 7 we cannot conclude whether or not a transition to chaos occurs in the spiking network. 
In simulations and in the linearized reduced model, we could however observe that the dis- 
tribution of pairwise correlations broadens when approaching the point of instability. Future 
work needs to examine this question in detail, e.g. by considering measures related to the Lya- 
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punov exponent. Recently developed semi-analytical theories accounting for nonlinear neural 
features [76] may be helpful to answer this question. The second limiting factor of the current 
theory is the use of a perturbative approach to quantify the response of the integrate-and-fire 
model. Although the steady-state firing rate of the network is found as the fixed point of the 
nonlinear self-consistency equation, the response to a synaptic fluctuation is determined up 
to linear order in the amplitude of the afferent rate fluctuation, which is only valid for suffi- 
ciently small fluctuations. For larger input fluctuations, nonlinear contributions to the neural 
response can become more important [28]. Also for strong synaptic coupling, deviations from 
our theory are to be expected. Thirdly, the employment of Fokker-Planck theory to determine 
the steady-state firing rate and the response to incoming fluctuations assumes uncorrected 
presynaptic firing with Poisson statistics and synaptic amplitudes which are vanishingly small 
compared to the distance between reset and threshold. For larger synaptic amplitudes, the 
Fokker-Planck theory becomes approximate and deviations are expected [66, 31, 58, 28]. This 
can be observed in Fig. 7A, showing a deviation between the self-consistent firing rate and 
the analytical prediction at about J ~ 1 mV. In this work, we obtained a sufficiently pre- 
cise self-consistent approximation of the correlation coefficient C/A by relating the random 
recurrent network of spiking neurons in the asynchronous irregular state to a reduced linear 
model which obeys the same relation between C and A up to linear order. This reduced linear 
model, however, does not predict the absolute values of the variance A and covariance C. 
The variance A of the LIF model, for example, is dominated by nonlinear effects, such as the 
reset mechanism after each action potential. Previous work [11, 9] has shown that the single 
spike-train statistics can be approximated in the diffusion approximation if the recurrent firing 
rate in the network is determined by mean-field theory. One may therefore extend our ap- 
proach and determine the integral auto-correlation function as A = isFF with the Fano factor 
FF (see [46]). For a renewal process and long observation times, the Fano factor is given 
by FF = CV 2 [15, 49]. The coefficient of variation CV can be obtained from the diffusion 
approximation of the membrane-potential dynamics ( [9] App.A.l). The covariance C can 
then be determined by (19). Another possibility is the use of a refractory-density approach 
[14, 43]. 

The spike-train correlation as a function of the time lag is an experimentally accessible 
measure. Future theoretical work should therefore also focus on the temporal structure of 
correlations in recurrent networks, going beyond zero-lag correlations [37, 57] and the integral 
measures studied in the current work. This would allow to compare the theoretical predictions 
to direct experimental observations in a more detailed manner. Moreover, the relative spike 
timing between pairs of neurons is a decisive property for Hebbian learning [26] in recurrent 
networks, as implemented by spike timing-dependent plasticity [5], and suspected to play a 
role for synapse formation and elimination [29]. 

The simulation experiments performed in this work revealed that the suppression of cor- 
relations is vulnerable to certain types of manipulations of the feedback loop. One particular 
biological source of additional variability in the feedback loop is probabilistic vesicle release at 
synapses [41]. In feedforward networks, such unreliable synaptic transmission has been shown 
to decrease the transmission of correlations by pairs of neurons [60]. Stochastic synaptic re- 
lease is very similar to the replacement of the population activity in the feedback branch by 
a rate modulated Poisson processes that conserves the population rate. In these simulations 
we observed an increase of correlations due to the additional noise caused by the stochastic 
Poisson realization. Future work should investigate more carefully which of the two opposing 
effects of probabilistic release on correlations dominates in recurrent networks. 

The results of our study do not only shed light on the decorrelation of spiking activity 
in recurrent neural networks. They also demonstrate that a standard modeling approach 
in theoretical neuroscience is problematic: When studying the dynamics of a local neural 
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network (e.g. a "cortical column"), it is a common strategy to replace external inputs to this 
neural population V by spike-train ensembles with some predefined statistics, e.g. by stationary 
Poisson processes. Most neural systems, however, exhibit a high degree of recurrence. Nonlocal 
input to the population V, i.e. input from other brain areas, therefore has to be expected 
to be shaped by the activity within V. The omission of these feedback loops can lead to 
qualitatively wrong predictions of the population statistics. The analytical results for the 
correlation structure of recurrent networks presented in this study provide the means to a 
more realistic specification of such external activity. 

4 Methods 

4.1 LIF network model 

In the present study, we consider two types of sparsely connected random networks: networks 
with purely inhibitory coupling ("I networks") and networks with both excitatory and inhibitory 
interactions ("El networks"). To illustrate the main findings of this study and to test the 
predictions of the linear model described in Sec. 4.2, both architectures were implemented as 
networks of leaky integrate-and-fire (LIF) neurons. The model details and parameters are 
reported in Tab. 1 and Tab. 2, respectively. All network simulations were carried out with 
NEST (www.nest-initiative.org). 

4.2 Linearized network model 

In this section we show how the dynamics of the spiking network can be reduced to an effective 
linear model whose fluctuations, by construction, fulfill the same relationship as the original 
system up to linear order. We first outline the major steps of this reduction, and then provide 
the formal derivation. 

We make use of the observation that the effect of a single synaptic impulse on the output 
activity of a neuron is typically small. Writing the response spike train of a neuron as a 
functional of the history of all incoming impulses therefore allows us to perform a linearization 
with respect to each of the afferent spike trains. Formally, this corresponds to a Volterra 
expansion up to linear order, the generalization of a Taylor series to functionals. In Sec. 4.3, 
we perform this linearization explicitly for the example of the LIF model. This determines how 
the linear response kernel depends on the parameters of the LIF model. The linear dependence 
on the input leads to an approximate convolution equation (31) linearly connecting the auto- 
and the cross-correlation functions in the network. As this equation is complicated to solve 
directly, we introduce a reduced linear model (35) obeying the same convolution equation. The 
reduced linear model can be solved by standard Fourier methods and yields an explicit form for 
the covariance matrix in the frequency domain (37). The diagonal and off-diagonal elements 
of the N = N E + N\ dimensional covariance matrix C{uS) in (56) correspond to the power- 
spectra of individual neurons and the cross-spectra of individual neuron pairs, respectively. As, 
in this linear approximation, both the auto- and the cross-covariances are proportional to the 
variance of the driving noise, the resulting correlation coefficients are independent of the noise 
amplitude (see Sec. 4.7). As shown in Sec. 2.2 and Sec. 2.3, the suppression of fluctuations in 
recurrent networks is most pronounced at low frequencies. It is therefore sufficient to restrict 
the discussion to the zero-frequency limit uj 0. Note that the zero- frequency variances and 
covariances correspond to the integrals of the auto- and cross-correlation functions in the time 
domain. In this limit, we may combine the two different sources of fluctuations caused by the 
spiking of the neurons and by external input to the network into a single source of white noise 
(39) with variance p 2 . 
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In general, the spiking activity Si(t) of neuron i at time t is determined by the entire 
history {s(t')\t f < t] of the activity of all neurons s = . . . , s N ) in the network up to time 
t. Formally, this dependence can be expressed by a functional 

Si {t) = G\[s{t% (26) 

The subscript t in G\ indicates that t f < t (causality). In the following, we will use the 
abbreviation G l t [s] = G z t [s(t f )]. The effect of a single synaptic input on the state of a neuron 
is typically small. We therefore approximate the influence of an incoming spike train on the 
activity of the target neuron up to linear order. The sensitivity of neuron z's activity to the 
input from neuron k can be expressed by the functional derivative of G\ with respect to input 
spike train s k \ 

= !5& e {Gl[s +e5{ °~ n ek] " Gl[s]) • (27) 

It represents the response of the functional to a single 5-shaped perturbation in input channel 
k at time t", normalized by the perturbation amplitude e. In (27), e k = (0, . . . , 0, 1, 0, . . . , 0) 
denotes the unity vector with elements e kk = 1 and e ki = for all i ^ k. By introducing 
the vector s^(t) = . . . , s k -i(t), 0, Sk+i(t), . . . , s N (t)) of spike trains with the fc-th 

component set to zero, G\[s\ can be approximated by 

sm-i:/ 9$ »<«•>*•■ pa) 
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Eq. (28) is a Volterra expansion up to linear order, the formal extension of a Taylor expansion 
of a function of N variables to a functional, truncated after the linear term. With the linearized 
dynamics (28), the pairwise spike-train cross-correlation function between two neurons % and 
j 7^ % is given by 

c ij( T ) = (^(t + ^Sjit))* 

n ^7,,^ r , , (29) 
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Note that (29) is valid only for positive time lags r > 0, because for r < a possible causal 
influence of si on Sj is not expressed by the functional. Here, (-) s denotes the average across 
the ensemble of realizations of spike trains in the stationary state of the network (e.g. the 
ensemble resulting from different initial conditions), and s(t) = s(t) — (s) s the centralized 
(zero mean) spike train. In the last line in (29), the average (-) s = ((') Sk ) s \s k is split into the 
average (') s \ Sk across all realizations of spike trains excluding s k and the average (-) Sk across 
all realizations of s k . Note that the latter does not affect the functional derivative because it 
is, by construction, independent of the actual realization of s k . A consistent approximation 
up to linear order is equivalent to the assumption that for all j the linear dependence of the 
functional on Sj is completely contained in the respective derivative with respect to sj (28). 
Dependencies beyond linear order include higher-order derivatives and are neglected in this 

approximation. This is equivalent to neglecting the dependence of S -%^my on Sj for any 
j ^ k. Hence, we can average the inner term over s k and Sj separately. In the stationary 
state, this correlation can only depend on t" — t and equals the auto- or the cross-correlation 
function: 



ttk(t" — t) for k — j 
c kj (t"-t) fork^j. 



26 



Tetzlaff et al. 



The pairwise spike-train correlation function is therefore given by 



N 



t+r 



Cij (r) = V / dt"( SG J+J [ f ) i a *f f0rk = J (Vr>0), 

Jl ^ ^7 \ M*") Is \c kj (t"-t) for ^ 

where we used the fact that (/[^]) s \ S/s = (f[sf,]) s for any functional / that does not depend 
on s k . The average of the functional derivative has the intuitive meaning of a response kernel 
with respect to a 5-shaped perturbation of input s k at time t" . Averaged over the realizations of 
the stationary network activity this response can only depend on the relative time t + r — t". In 
a homogeneous random network, the input statistics (number of synaptic inputs and synaptic 
weights) and the parameters of the internal dynamics are identical for each cell, so that the 
temporal shape h{t) of the response kernel can be assumed to be the same for all neurons. 
The synaptic coupling strength from neuron k to neuron i determines the prefactor w ik : 

^r-f) s (M^. (30) 

In this notation, the linear equation connecting the auto-correlations a k and the cross-correlations 
Cij takes the form 

c l3 {r) =f>* / dth(r - t) H £) I " \ = { (Vr > 0). (31) 

-oo v 

Our aim is to find a simpler model which is equivalent to the LIF dynamics in the sense 
that it fulfills the same equation (31). Let's u(t) denote the vector of dynamic variables of 
this reduced model. Analog to the original model, we define the cross-correlation for i ^ j 
and r > as 

c U r ) = + r )uj{t))u 

= (Ll +T [u}u j (t)) u . (32) 

The simplest functional L\[u] consistent with equation (31) is linear in u. Since we require 
equivalence only with respect to the ensemble averaged quantities, i.e. c^-(t) = c^(t), the 
reduced activity and therefore L\[u] can contain a stochastic element which would disappear 
after averaging. The linear functional 

N * 

Ui(t)=L\[u] = "^w ik h{t-t')u k {t')dt' + Zi(t) (33) 

*=* -oo 

with a pairwise uncorrected, centralized white noise zi(t) ((zi(t + r)z k (t)) z = p 2 z SijS(r)) 
fulfills the requirement, since for r > and i ^ j 
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Cy(r) = «* + r)fi j (t)> JB = 2^w ik I h(t + r-t , )(u k (t f )u j (t)) z dt f + ( Zi (t + r) Uj(t)) z 
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This equation has the same form as (31), so both models, within the linear approximation, 
exhibit an identical relationship between the auto- and cross-covariances. The physical meaning 
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of the noise z(t) is the variance caused by the spiking of the neurons. The auto-correlation 
function of a spike train of rate v has a 5-peak of weight v. The reduced model (33) exhibits 
such a 5-peak if we set p\ — v. A related approach has been pursued before (see Sec. 3.5 
in [11]) to determine the auto-correlation of the population averaged firing rate. This similarity 
will be discussed in detail below. 

So far, we considered a network without external drive, i.e. all spike trains s(t) originated 
from within the network. If the network is driven by external input, each neuron receives, 
in addition, synaptic input y { from neurons outside the network. We assume uncorrected 
external drive (yi(t + r))yj(t)) = p 2 8ij8{r). In the reduced model, this input constitutes a 
separate source of noise: 



N 



Ui(t) = L\[u,y) = ^2w ik (u k * h) + (yi * h iy ) + z^t). (34) 

k=i 

Here, (f*g)(t) = dt f f(t f )g(t — t') denotes the convolution and h iy (t) the response kernel 
with respect to an external input. For simplicity, let's assume that the shape of these kernels is 
identical for all pairs of pre- and postsynaptic sources, i.e. h ix (t) = h(t). If we further absorb 
the synaptic amplitude of the external drive in the strength of the noise p y , the linearized 
dynamics (34) can be written in matrix notation 

u(t) = {[Wu + y] * h)(t) + z(t) (35) 

with W = {wij}. The reduced model (35) can be solved directly by means of Fourier 
transform: 

U(uj) = [1-WH(uj)]-\H(uj)Y(uj) + Z(uj)). (36) 

The full covariance matrix follows by averaging over the sources of noise Z and Y as 

C» = (U(lo)U t (-lu)) zx 

= {pl + \H{u)\ 2 pl)[l-WH{u:)}- 1 [l-W T H{-u:)}-\ (37) 

The diagonal elements of C u represent the auto-covariances, the off-diagonal elements the 
cross-covariances. Both are proportional to the driving noise p 2 z + \H(uj)\ 2 p 2 y . This is consistent 
with (31) which is a linear relationship between the cross- and auto-covariances. 

For networks which can be decomposed into homogeneous subpopulations, the TV dimen- 
sional system (35) can be further simplified by population averaging. Consider, for example, 
a homogeneous random network with purely inhibitory coupling. Assume that the neurons are 
randomly connected with probability e and coupling strength — w < 0. The average number 
of in/outputs per neuron (in/out-degree) is thus given by K = eN . By introducing the pop- 
ulation averaged external input y(t) = [yi(t)], the averaged spiking noise z(t) = [^(£)], 
and the effective coupling strength w = Kw, the dynamics of the population averaged activity 
becomes 



u(t) = E z [m(t)] 



yz e * m u 3 + e * n 



* h (t) + Ei [Zi(t)] = ([-wu+y]*h)(t)+z(t) . 



(38) 

Here we assumed that E^ [wij] is independent of the presynaptic neuron j and can be replaced 
by —ew = —w/N. Note that this replacement is exact for networks with homogeneous 
out-degree, i.e. if the number of outgoing connections is identical for each neuron j. For 
large random networks with binomially distributed out-degrees (e.g. Erdos-Renyi networks or 
random networks with constant in-degree), (38) serves as an approximation. 
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To relate our approach to the treatment of finite-size fluctuations in [11], consider the 
population-averaged dynamics (38) of a single population with mean firing rate v. We set 
p 2 z = v for all single neuron noises zi in order for the reduced model's auto-covariances to 
reproduce the 5-peak of the spiking dynamics. In the population averaged dynamics, this leads 
to the variance of the noise z(t) given by (z(t + r) z(t)) = ^p 2 z 5(r) = jj5(r). This agrees 
with the variance of the population rate in [11]. Therefore, the dynamics of the population 
averaged quantity u in (38) agrees with the earlier definition of a population averaged firing 
rate s(t) = ^ J2i s i(t) f° r the spiking network [11]. 

In equation (38), two distinct sources of noise appear: The noise due to external uncorre- 
cted activity y and the noise z which is required to obtain the 5-peak of the auto-correlation 
functions of the reduced model. The qualitative results of Sec. 2.2 and Sec. 2.3, however can 
be understood with an even simpler model. As we are mainly concerned with the low-frequency 
fluctuations, we only need a model that has the same limit uj — » 0. As we normalized the ker- 
nel so that H(0) = 1 we can combine both sources of noise and require X(0) = Y(0) + Z(0) 
in (36) in the zero frequency limit. Hence, in Sec. 2.2 and Sec. 2.3, we consider the model 

r(t) = ([-Wr + x]*h){t) (39) 

with a pairwise uncorrected centralized white noise E x [xi(t + r)xj(t)] = p 2 SijS(r) to explain 
the suppression of fluctuations at low frequencies. 

As a second example, consider a random network composed of an excitatory and an in- 
hibitory subpopulation £ and X with population sizes N E = \£\ and N\ = \X\ = jN E , 
respectively. Assume that each neuron receives excitatory and inhibitory inputs from £ and 
X with coupling strengths w > and —gw < 0, respectively, and probability e, such that 
the average excitatory and inhibitory in /out-degrees are given by K = eN and jK, respec- 
tively. The dynamics of the subpopulation averaged activities u(t) = {u E {t) ) u\{t)) T is given 
by (35) with subpopulation averaged noise y(t) = (yE(t),y\(t)) T and z(t) = (z E (t), z\(t)) T 
and effective coupling 

W = w 

Here, w = Kw denotes the effective coupling strength, g = the effective balance parameter 
and 2/e/i (*) = E ie£ /x [Vi{t)] and z E /\(t) = E ie£ /x [zi{t)] the (sub)population averaged external 
and spiking sources of noise, respectively. Again, the reduction of the TV-dimensional linear 
dynamics to the two-dimensional dynamics (40) is exact if the out-degrees are constant within 
each subpopulation. As before, both sources of noise can be combined into a single source of 
noise, if the we are only interested in the low-frequency behavior of the model, leading to the 
dynamics (39) with the effective coupling (40). 

The linear theory is only valid in the domain of its stability, which is determined by the 
eigenvalue spectrum of the effective coupling matrix W. For random coupling matrices, the 
eigenvalues are located within a circle with a radius equal to the square root of the variance of 
the matrix entries [56] ^/Var \w^\ = w^Ne(l — e)(l + 7# 2 ). Writing the effective dynamics 
for the exponential kernel as a differential equation = (W — l)r + x(t), the eigenvalues of 
the right hand side matrix W — 1 are confined to a circle centered at —1 in the complex plain 
with radius ^/Var Given Var [wij] > 1, eigenvalues might exist which have a positive 

real part, leading to unstable dynamics. This condition is indicated by the vertical dotted lines 
in Fig. 6A-F and Fig. 7B-D near J = 2.8 mV. Beyond this line, the linear model predicts an 
explosive growth of fluctuations. In the LIF-network model, an unbounded growth is avoided 
by the nonlinearities of the single-neuron dynamics. 



(40) 
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4.3 Response kernel of the LIF model 

We now perform the formal linearization (30) for a network of TV LIF neurons i — 1, . . . , N. A 
similar approach has been employed in previous studies to understand the population dynamics 
in these networks [11, 9]. We consider the input J^ . JijSj(t) received by neuron i from the 
local network, where Sj denotes the spike train of the neuron j projecting to neuron i with 
synaptic weight J^. Given the time dependent firing rate i/j(t) of each afferent, and assuming 
small correlations and small synaptic weights, the total input to neuron i can be replaced by 
a Gaussian white noise with mean ^(t) and variance of (i), 

3 

= ^E4^ ( 41 ) 

3 

where j sums over all synaptic inputs. Jy G { J, — g J} denotes the amplitude of the postsy- 
naptic potential evoked by synapse j i. r m is the membrane time constant of the model. 
In the stationary state, the firing rate of each afferent is well described by the constant time 
average Vj = E t [uj]- The working point at which we perform the linearization of the neu- 
ral response (30) is then given by analog equations as (41), resulting in a constant mean 
fli = T m J2j JijVj and variance of = r m ^ JfjVj. If the amplitude of each postsynaptic po- 
tential is small compared to the distance of the membrane potential to threshold, the dynamics 
of the LIF model can be approximated by a diffusion process, employing Fokker-Planck theory 
[59]. The stationary firing rate of the neuron is then given by [65, 11, 9] 

v^ifli, Gi) = r ref + ^r rn (F(y ^) - F(y r J) 
with 

y 

F(y) = J f(y)dy /(y) = e^(erf(y) + l) 

$ — fli Preset — fli 

VOj = — Vr,i = = , 

with the reset voltage V resetl the threshold voltage 6 and the refractory time r re f. In homo- 
geneous random networks, the stationary rate (Fig. 7A) is the same for all neurons. It is 
determined in a self-consistent manner [9] as the fixed point of (42). The stationary mean 
and variance of are determined by the stationary rate. To determine the kernel (30) we need 
to consider how a 5-shaped deflection in the input to this neuron at time point t f affects its 
output up to linear order in the amplitude of the fluctuation. In the stationary state, we may 
set t' = 0. It is therefore sufficient to focus on the effect of a single fluctuation 

s k (t) =aS(t). (43) 

We therefore ask how the density of spikes per time vi(t) = {G\[s]) s \ Sk of neuron i, averaged 
over different realizations of the remaining inputs to neuron i, changes in response to the 
fluctuation (43) of the presynaptic neuron k in the limit of vanishing amplitude a. This kernel 
w ik h (30) is identical to the impulse response of the neuron and can directly be measured in 
simulation by trial averaging over many responses to the given 5-deflection (43) in the input 
(see Fig. 10A). For the theory of low-frequency fluctuations, we only need the integral of the 
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kernel, also known as the DC susceptibility, 



oo 

w ' k = < 44 > 



_ j. Vj(fii + SfJLj, Gj + S(Tj) ~ VjjjLi, Gj) 
a^O a 

The second equality follows from the equivalence of the integral of the impulse response and the 
step response in linear approximation [16, 28]. Following from (41), both mean and variance 
are perturbed as 5 fa = ar m Jik and 5 a? = ar m J^ k in response to a step a in the afferent rate Vy 
Moreover, we used the chain rule Sdi = ^rSaf. The variation of the afferent firing rate hence 
co-modulates the mean and the variance and both modulations need to be taken into account 
to derive the neural response [11]. Although the finite amplitude of postsynaptic potentials 
has an effect on the response properties [28, 58], the integral response is rather insensitive to 
the granularity of the noise [28]. We therefore employ the diffusion approximation to linearize 
the dynamics of the LIF neuron around its working point characterized by the mean and 
the variance of of the total synaptic input. In (44), we evaluate the partial derivatives of ^ 

with respect to fa and o\ using (42). First, observe that by chain rule |^ = —vf^j—. We 
then again make use of the chain rule = l^^ 21 + Analog expressions hold 

for the derivative with respect to o^. The first derivative yields = y/^r m f{ye,i)i the one 
with respect to y r ^ follows analogously, but with a negative sign. We further observe that 
^0 = ^ and ^ = ZZ ^ A with y A G {y r ,i,yo,i}- Taken together, we obtain the explicit result 
for (44) 1 

Wik = (viTmfVn^ (f(yo,i)(l + £r-yo,i) ~ /(2/r,i)(l + ^Vr,i) J ■ (45) 

Gi \ ZGi ZGi J 

Note that the modulation of \i{ results in a contribution to that is linear in J^, whereas the 
modulation of Oi causes a quadratic dependence on J^. This expression therefore presents an 
extension to the integral response presented in [16, 27]. Fig. 10B shows the comparison of the 
analytical expression (45) and direct simulation. The agreement is good over a large range of 
synaptic amplitudes G [—4, 4] mV in the case of constant background noise caused by small 
synaptic amplitudes (here 0.1 mV for excitation and — 0.4mV for inhibition). For background 
noise caused by stronger impulses, the deviations are expected to grow [28]. 



4.4 Population-activity spectra in the linear model: feedback vs. feed- 
forward scenario 

The recurrent linear neural dynamics defined in the previous section is conveniently solved in 
the Fourier domain. The driving external Gaussian white noise X is mapped to the response 
R(cj) = T(uj)X(uj) by means of the transfer matrix T(cj). According to (39), it is given by 
T{uS) = H{uS) (1 — H{uS) W)~ x . The covariance matrix in the frequency domain, the spectral 
matrix, thus reads 

Crr(u) = E x [R{u)R(u)*] = 2»p 2 lTH* , (46) 

where we used E x [XX*] = p 2 l and the expectation operator E x [] represents an average over 
noise realizations. To identify the effect of recurrence on the network dynamics, we replace 



31 



Decorrelation by inhibitory feedback 



the local feedback input by a feedforward input Q with spectral matrix Cqq. The resulting 
response firing rate is given by R = H(WQ + X). Assuming that the feedforward input Q 
is uncorrected to the external noise source X (Cq X = 0) yields a response spectrum 

H\ 2 (WC QQ W* + p 2 l) . (47) 



RR* 



4.5 Population-activity spectrum of the linear inhibitory network 

In the Fourier domain, the solution of the mean-field dynamics (38) of the inhibitory network 

is R(lj) = H(lj)X(cjj)/(1 + wH(uj)). The power-spectrum C rr (uj) = E x [R(u;)R(lj)*] hence 
becomes 

c «^> - ^rSr 2 ' < 48 > 

using the spectrum of the noise E x [X(cj)X(lj)*] = p 2 . 

We compare this power-spectrum to the case where the feedback loop is opened, i.e. where 
the recurrent input is replaced by feedforward input with unchanged auto-statistics Cqq(uS) = 
Crr{u), but which is uncorrected to the external input Cq X (u) = 0. The resulting power- 
spectrum is given by (47) as = \H\ 2 (w 2 C RR + p 2 ). 



4.6 Population-activity spectra of the linear excitatory-inhibitory net- 
work 

In a homogeneous random network of excitatory and inhibitory neurons, the population aver- 
aged activity (40) can be solved in the Schur basis (9) introduced in Sec. 2.3 

R-(u>) = H(uj)X_(uj), 
with w + = —w(l — g) and u>ff = w{l + g). The power of the population rate therefore is 
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C r+r+ {uj) = 

C R _ R _{u) = - (1 + 7 l )- 

The fluctuations of the excitatory and the inhibitory population follow as 



(50) 



R ^ = ^TT§^k x ^ (51) 



V2 

So the power-spectra are 
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\H(u)\ 2 p 2 _ 
Cr eRi {u) = w 
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Replacing the recurrent input of the sum activity R+ by activity Q+ with the same auto- 
statistics, but which is uncorrelated to the remaining input into R+ (Fig. 5D') results in the 
fluctuations 

(53) 



R + (co) = H(u) \^-w + Q + (cu) + w FF R-(co) + X + (co) ) , 
iL(u) = H(uj)X_(uj). 
The power-spectrum of the sum activity therefore becomes 



C R+R+ (u;) = \H(u)f 



wlC 



R-\-R-\- 



2N f 



(]H(u)w FF + 1| 2 + r y~ 1 \H(uj)w FF - 1| 2 ) 



(54) 



If, alternatively, the excitatory and the inhibitory feedback terms i? E and R\ are replaced by 
uncorrelated feedforward input Qe and Q\ with power-spectra Cr e r e and C R[R[ (Fig. 5C,D), 
the spectrum of the sum activity reads 



2w 2 (C RERE (u)+fC R M) + ^-(1 + 7" 1 ) 



(55) 



The limit (14) for inhibition dominated networks with g > 1 can be obtained from this and 
the former expressions by taking H(0) = 1 and assuming strong coupling w ^> 1. 



4.7 Population averaged correlations in the linear El network 

In this subsection, we derive a self-consistency equation for the covariances in a recurrent 
network. We start from (37) (we drop the superscript u of C u for brevity) multiply by 
1 — WH(uj) from left and its transpose from right to obtain 

C(uj) = H(uj)WC(uj) + C(uj)H(-uj)W t (56) 
- \H(uj)\ 2 WC(uj)W t + l(\H(u)\ 2 p 2 y + p 2 z ) 

We assume a recurrent network of excitatory and N\ inhibitory neurons, in which each 
neuron receives K excitatory inputs of weight w and inhibitory inputs of weight —gw 
drawn randomly from the presynaptic pool of neurons. To obtain a theory for the variances 
and covariances at zero frequency (with H(0) = 1) we may abbreviate p 2 z (0) + \H (Q)\ 2 p 2 y (Q) 
by p 2 (0). For a population averaged theory, we need to replace in (56) the variances A { of an 
individual neuron by the population average and replace the covariance for a given pair of 
neurons by the average over pairs that are statistically equivalent to For a pair 

(i, j) of neurons we will show that the set of equivalent pairs depends on the current realization 
of the connectivity since unconnected pairs are not equivalent to connected ones. Therefore 
it is necessary to first average the covariance matrix over statistically equivalent neuron pairs 
given a fixed connectivity and to subsequently average over all possible realizations of the 
connectivity. The latter will be denoted as []. For compactness of the notation, first we 
perform the averaging for the general case, where neuron i belongs to population x and neuron 
j to population y. We denote by X y the sets of neuron indices belonging to populations x 
and y, respectively. Subsequently replacing x and y by all possible combinations {E, I}, 

we obtain the averaged self-consistency equations for the network. We denote the number of 
incoming connections to a neuron of type x from the population of neurons of type y as K xy 
and the strength of a synaptic coupling as w xy . Rewriting the self-consistency equation (56) 
explicitly with indices yields 

Cij = ^2 w ikCkj + ^2 WjkC ik - ^2 w ^Cki^ji + p 2 $ij • (57) 

k k k 
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The last equation shows that for a connected pair (i, j) of neurons (wij ^ or wji ^ 0) either 
of the first two sums contains a contribution WijCjj or w^Ca proportional to the variance of 
the projecting neuron. We therefore need to perform the averaging separately for connected 
and for unconnected pairs of neurons. We use the notation 



C x ^y — E\4/ 



(58) 



for the average covariance over pairs of neurons of types {E, 1} with a connection from 

neuron j G y to neuron z G X, where N pa \ rs ^ x ^ y is the number neuron pairs connected in this 
way. An arrow to the right, i — » j, denotes a connection from neuron z to neuron j. Note 
that we use the same letter C for the population averaged covariances and for the covariances 
of individual pairs. The distinction can be made by the indices: z, j, k,l throughout indexes a 
single neuron, u,v,x,y,z identifies one of the populations {E, I}. We denote the covariance 
averaged over unconnected pairs as 



x^y 



~-W 



pairs, x^y 



E 



(59) 



We further use 



An 



~-w 



.v,X r " 



(60) 



for the integrated variance averaged over all neurons of type x. Connected and the unconnected 
averaged covariances differ by the term proportional to the variance of the projecting neuron, 
as mentioned above 



a 



x^y 



a 



x^y 



^ x^-y ^xyAy 
C x ^±y H~ ^yxA x • 



(61) 



As a consequence, we can express all quantities in terms of the averaged variance (60) and the 
covariance averaged over unconnected pairs (59). We now proceed to average the integrated 
variance over population x. Since there are no self-connections in the network, we do not need 
to distinguish two cases here. Replacing Cu on the right hand side of (60), the first term of 
(57) contributes 
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^ ^ K xz w xz C z ^, x 
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-W 



x iex \kes kex / 



(62) 



yZ K xz w xz (C z ^ x + w xz A z ) 
*e{E,i} 



From the second to the third step we used that the sum over k (I) yields non-zero contributions 
only if neuron k (I) connects to neuron z. This happens in K x e (K x \) cases with the coupling 
weight w x e {w x \). Therefore the covariance averaged over connected pairs appears on the 
right hand side. In the last line we used the relation (61) to express the connected covariance 
in terms of the variance and the covariance over unconnected pairs. The second term in (60) 
is identical because of the symmetry dk = C^. Up to here, the structure of the network only 
entered in terms of the in-degree of the neurons. The contribution of the third term follows 



34 



Tetzlaff et al. 



from a similar calculation 
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From the second to the third step we assumed that among the K XZ K XW pairs of neurons 



fcG2,lG)V projecting to neuron z, the fraction has a connection k —> 1. These pairs 
contribute with the connected covariance. The connections in opposite direction contribute the 
other term of similar structure. We ignore multiple and reciprocal connections here, assuming 
the connection probability is low. We introduce the shorthand C xy for the covariance averaged 
over all neuron pairs including connected and unconnected pairs 
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(64) 



This is the covariance which is observed on average when picking a pair of neurons of type 
x and y randomly. In this step, beyond the in-degree, the structure of the network entered 
through the expected number of connections between two populations. Taken all three terms 
together, we arrive at 
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The averaged covariances follow by similar calculations. Here we only need to calculate the 
average over unconnected pairs given by (59), because the connected covariance follows 
from (61). The first sum in (57) contributes 



a 



def 



xy,A 



-W 



1 



_^pairs,xt% ie x,jey,i^j,ij£j keSWX 

^ ^ K XZ W XZ C Z y ) 

*€{E,I} 

where due to the absence of a direct connection between i and j, the term linear in the 
coupling and proportional to the variance is absent. From the symmetry Cm = Cm it follows 
that the second term corresponds to an exchange of x and y in the last expression. The third 
sum in (57) follows from an analog calculation as before 
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In summary, the contributions from (66) and (67) together result in the self-consistency equa- 
tion for the covariance 



a 



x^y 



C X y J\ + C t 



yx,A 



a 



xy,B 



(68) 



We now simplify the expressions by assuming that the in-degree of a neuron and the incoming 
synaptic amplitudes do not depend on the type of the neuron, i.e. that excitatory and inhibitory 
neurons receive statistically the same input. Formally this means that we need to replace K xy 
by K yi the number of incoming connections from population y and w xy by w y , the coupling 
strength of a projection from a neuron of type y. The covariance C x ^± y then has two distinct 
contributions, C xy ^ sep that depends on the type of neurons x,y, and C com that does not. 
In particular C xy ^ and C xy ^ ou do not depend on x,y and we omit their subscripts in the 
following. The variances fulfill 
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the covariances satisfy 
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(70) 



The disjoint part C xy:Sep determines the difference between the covariances for pairs of neurons 
of different type. Using the parameters K E = K, K\ = jK, w E =w,w\ = —gw, the explicit 
form is 



CEE.sep = 2Kw 2 (l- 7 g)^A E + 2KwC E¥ , E -2 ig KwC E ^ 
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(71) 



Therefore, also the covariances in the network obey the relation 



E^E 



C\$>\ — 2C\ 



E^h 



(72) 



i.e. the mixed covariance can be eliminated and is given by the arithmetic mean of the 
covariances between neurons of same type. In matrix representation with the vector Q = 
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(A E , Aj, Ce^e, C\^±\), the self-consistency equation is 
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The self consistent covariance can then be obtained by solving the system of linear equations 

(/ - M dis - M com )Q = p 2 (l, 1, 0, 0) T . (76) 

The numerical solution shows that the variances for excitatory and inhibitory neurons are 
approximately the same, as depicted in Fig. 6A. In the following we therefore assume A E = 

A\ = A and then solve (76) for the covariances. With the abbreviation G = + 

the third and fourth line yields the equation for the covariances 
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The structure of the equation suggests to introduce the linear combination m = Cee — igC\\ 
which satisfies 
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We solve (77) for Ce^e and C\^±\ and insert (78) for m to obtain the covariances as 
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The covariance C x ^ y between unconnected neurons can be related to the covariance between 
the incoming currents this pair of neurons receives. Expressing the self-consistency (68) in 
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terms of the covariances averaged over connected and unconnected pairs (64) uncovers the 
connection 

C&y = Yl K z w z (C zx + C zy ) (80) 
ze{E,\} 
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- XI w2 z~^ Az ~ XI K u K v w u w v C uv 

2€{E,I} 2 u,ve{E,\} 

= Kw [C Ex + C Ey - 13 (Q x + Q y )} 



(Kw) 



\2 



j^A E + Ml A, + C EE - 2 19 C B + ( ig ) 2 C n 



This self-consistency equation yields the argument, why the shared-input correlation Cg{J ared 
(19) cancels the contribution C^ on (20) due to spike-train correlations in the covariance to the 
input currents (see Fig. 6C,D). Rewriting (80) in terms of these quantities results in 



[C Ex + C Ey - 19(C {X + Q y )] (81) 



Kw 

= Kw[CZ r J(Kw) 2 + C? on /(Kw) 2 ] . 

If a self-consistent solution with small correlation \C xy \ < e exists, the right hand side 

of (81) must be of the same order of magnitude. The right hand side of this equation has a 
prefactor Kw which typically is 3> 1 (for the parameters in Fig. 6, Kw becomes larger than 
1 for w > 10" 3 ). The first term in the bracket is proportional to the contribution of shared 
input, the second term is due to correlations among pairs of different neurons. Each of these 
terms is of order s. Due to the prefactor Kw, however, the sum of the two terms needs to be 
of order e/(Kw) to fulfill the equation. Hence, the terms must have different signs to cause 
the mutual cancellation. 

To illustrate how the correlation structure is affected by feedback, let us now consider the 
case where the feedback activity is perturbed ("feedforward scenario"). We start from (47) 
and, again, only consider the fluctuations at zero frequency, 

C M (0) = WC QQ W T + lp 2 . (82) 

First, we consider a manipulation that preserves the single-neuron statistics A El A\ and the 
pairwise correlations C E e, Q\ within each subpopulation, but neglects correlations C E \ between 
excitatory and inhibitory neurons. Formally, this corresponds to the block diagonal correlation 
matrix 



^ SijA E + (1 - S i:j )C EE ij e£ 

iij 



Ma ^SijA\ + (1 - Sij)C\\ i,j el. K ) 

Here, we have replaced the individual entries of the correlation matrix by the corresponding sub- 
population averaged correlations. The calculation of the response auto- and cross-correlation A 
and C is similar as for the expressions (63) and (67), with the difference that terms containing 
C E \ are absent: 

A = Kw 2 (A E + ~ig 2 A\) +c + p 2 

^2 Me , , x2^' 



c = {Kw) [w E + {7g) nJ +c (84) 

with c = (Kw) 2 (C EE + (igfCn) . 

As an alternative type of feedback manipulation, we assume that all correlations are equal, 
irrespective of the neuron type. To this end, we replace all spike correlations by the population 
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average C = (NlC EE + N^C n + 2N E N i C Ei )/{N E + N i ) 2 = (W E C EE + W,qi)/(JV E + JV|). Thus, 
the covariance matrix reads 

C QQij = S ij A + (l-S ij )a (85) 
The calculation follows the one leading to the expressions (63) and (67) and results in 

A = w 2 K{l + ig 2 )A+(wK) 2 {l- ig ) 2 C + p 2 (86) 
C = w 2 K 2 (^+ { ^ 
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Figure 6. Dependence of population averaged correlations and population-rate fluctuations on 
the effective coupling w = Kw in a linearized homogeneous network with excitatory-inhibitory 
coupling. A: Spike-train variances A E (black) and A\ (gray) of excitatory and inhibitory 
neurons. B: Spike-train covariances Cee (black solid), Ce\ (dark gray solid) and C\\ (light gray 
solid) for excitatory-excitatory, excitatory-inhibitory and inhibitory-inhibitory neuron pairs in 
the recurrent network, respectively, and shared-input contribution Qh ared (black dotted curve; 
'feedforward case'). C: Decomposition of the total input covariance C m (light gray) into 
shared-input covariance QK ared (black) and weighted spike-train covariance C™ Qrr (dark gray). 
Covariances in A, B and C are given in units of the noise variance p 2 . D: Input-correlation 
coefficient C m /A m in the recurrent network (black solid curve). In the feedforward case, the 
input-correlation coefficient is identical to the network connectivity e (horizontal dotted line). 
E: Spike-train correlation coefficients Cee/A e (black), Ce\/VA^A\ (dark gray) and C\\/A\ 
(solid light gray curve) for excitatory-excitatory, excitatory-inhibitory and inhibitory-inhibitory 
neuron pairs, respectively. Thick dashed curves represent approximate solutions assuming 
A E = A\. F: Low-frequency (LF) power ratios a (black), a E (dark gray), ct\ (solid light gray) 
for the population rate r(t) and the excitatory and inhibitory subpopulation rates rt(t) and 
r\(t), respectively. The LF power ratio represents the ratio between the LF spectra in the 
recurrent network and for the case where the feedback channels are replaced by feedforward 
input with Ce\ = (cf. Fig. 5C). Thick dashed curves in F show power ratios obtained by 
assuming that the auto-correlations are identical in the feedback and the feedforward scenario 
(see main text). Vertical dotted lines mark the stability limit of the linear model (see Sec. 4.2). 
A-F: K = 1000, e = 0.1, 7 = 1/4, g = 6, g = -yg = 3/2, N = K(1 + 7 )/e = 12500. 
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Figure 7. Comparison between predictions of the linear theory (thick gray curves) and direct 
simulation of the LIF-network model (symbols and thin lines). Dependence of the spike- 
train and population-rate statistics on the synaptic weight J (PSP amplitude) in a recurrent 
excitatory-inhibitory network ('feedback system', 'FB') and in a population of unconnected 
neurons receiving randomized feedforward input ('feedforward system', 'FF') from neurons 
in the recurrent network. Average presynaptic firing rates and shared-input structure are 
identical in the two systems. In the FF case, the average correlations between presynaptic 
spike-trains are homogenized (i.e. Cee = Cei = C\\) as a result of the random reassignment 
of presynaptic neuron types. The mapping of the LIF dynamics to the linear reduced dynam- 
ics (Sec. 4.3) relates the PSP amplitude J to the effective coupling strength w(J) by (45), 
as shown in Fig. 10B. A: Average firing rates i/ in the FB (black up-triangles: excitatory 
neurons; gray down-triangles: inhibitory neurons) and in the FF system (open circles). An- 
alytical prediction (42) (gray curve). B: Spike-train correlation coefficients Cee/ At (black 
up-triangles), Ce\/\/AeA\ (gray squares) and C\\/A\ (gray down-triangles) for excitatory- 
excitatory, excitatory-inhibitory, and inhibitory-inhibitory neuron pairs, respectively, in the FB 
system. Analytical prediction (19) (gray curves). Spike-train correlation coefficient C/A 
(open circles) in the FF system with homogenized presynaptic spike-train correlations. Analyt- 
ical prediction (86) (underlying gray curve). C: Shared-input (CgJJ ared ; black up-triangles) and 
spike-correlation contribution C™ on (FB: gray down-triangles; FF: open circles) to the input 
correlation C m (normalized by ^ AeA\). Analytical predictions (20). D: Low-frequency (LF) 
power ratio of the compound activity. Vertical dotted lines in A-D mark the stability limit of 
the linear model (see Sec. 4.2). TV = 12500, K = 1000, 7 = 1/4, g = 6. Size of postsynaptic 
population in the FF case: M = 2000. Simulation time: T = 100s. 
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Figure 8. Amplification of population-rate fluctuations by different types of feedback manipu- 
lations in a random network of excitatory and inhibitory LIF neurons (simulation results). Top 
row (A-C): Unperturbed feedback (FB; black), shuffling of spike-train senders across entire 
network (ShufflD; dark gray) and within each subpopulation (E,l) separately (Shuff2D; light 
gray). Bottom row (D-F): Unperturbed feedback (FB; black), replacement of spike trains by 
realizations of inhomogeneous (Poissl; dark gray) and homogeneous Poisson processes (PoissH; 
light gray). In the Poissl (PoissH) case, the (time averaged) subpopulation rates are approxi- 
mately preserved. A,D: Compound power-spectra Cqq of input spike-train ensembles. B,E: 
Power-spectra of population-response rates. C,F: Low-frequency (LF; 1-20 Hz) power 
ratio a (increase in LF power relative to the unperturbed case [FB]; logarithmic scaling). Note 
that in A, the compound-input spectra (FB, ShufflD, Shuff2D) are identical. In D, the input 
spectra for the intact recurrent network (FB) and the inhomogeneous-Poisson case (Poissl) 
are barely distinguishable. See Tab. 1 and Tab. 2 for details on the network model and param- 
eters. Simulation time T = 100s. Single-trial spectra smoothed by moving average (frame 
size 1 Hz). 
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Figure 9. Recurrent network dynamics stabilizes dynamics of embedded synfire chains. Spik- 
ing activity in a synfire chain (L = 10 layers, layer width b = 50) receiving background 
input from an excitatory-inhibitory network (A, cf. Fig. 1C) or from a finite pool of excitatory 
and inhibitory Poisson processes (B, cf. Fig. ID). Average input firing rates, in-degrees and 
amount of shared input are identical in both cases. Neurons of the first synfire layer (neuron 
ids 1, ...,&) are stimulated by current pulses at times t = 500 and 1000 ms. Each neuron 
in layer k e [2,L] receives inputs from all b neurons in the preceding layer k — 1 (synaptic 
weights J s f c = 0.8 mV, spike transmission delays rf s f c = 2 ms), and K — b and excitatory 
and inhibitory background inputs, respectively, randomly drawn from the presynaptic popula- 
tions. Neurons in the first layer k — 1 receive K and excitatory and inhibitory background 
inputs, respectively. Note that there is no feedback from the synfire chain to the embedding 
network. See Tab. 2 for network parameters. 




Figure 10. A: Firing-rate deflection w ik h(t) of a LIF neuron caused by an incoming spike 
event of postsynaptic amplitude = 0.6 mV. B: Integral = J °° h(t) dt of the 
firing rate deflection shown in A as a function of the postsynaptic amplitude J ik (simulation: 
black dots; analytical approximation (45): gray curve). The neuron receives constant synaptic 
background input with J = 0.1 mV, g = 4, and rates v E = 5960^, v\ = 1190 \ resulting in 
a first and second moment (42) fii = 12 mV and Ui = 5 mV. Simulation results are obtained 
by averaging over 1000 trials of 10s duration each with 25000 input impulses on average. For 
further parameters of the neuron model, see Tab. 1 and Tab. 2. 
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A 




Model summary 


Populations 


one (inhibitory network) or two (excitatory-inhibitory network) 


Connectivity 


random, fixed in-degrees 


Neuron 


leaky integrate- 


and-fire (LIF) 


Synapse 


current based, delta-shaped postsyn. currents with constant amplitudes 


Input 


uncorrected Gaussian white noise currents 


B 




Populations 


Inhibitory network 


Name 


Elements 


Size 


1 


LIF 


N = K/e 


Excitatory-inhibitory network 


Name 


Elements 


Size 


E 
1 


LIF 
LIF 


N E = K/e 

AT, = 7 iV E = ^K/e 


c 




Connectivity 


Inhibitory network 


Source 


Target 


Pattern 


1 


1 


random convergent K 1, delay d, weight — J 


Excitatory-inhibitory network 


Source 


Target 


Pattern 


E 
E 
1 
1 


E 
1 

E 
1 


random convergent K —> 1, delay d, weight J 
random convergent K —> 1, delay d, weight J 
random convergent — » 1, delay d, weight — gJ 
random convergent 1, delay d, weight — gJ 


D 




Neuron 


Type 

Description 


Leaky integrate-and-fire (LIF; [80]) 

Dynamics of membrane potential V*(t) (i G [1, AT]): 

- Spike emission at times t\ with Vi(t\) > 6 

- Subthreshold dynamics: 




r m Vi = - 


-Vi + RMt) ifVfc: ti (4,4 + r ref ] 




- Reset + refractoriness: Vi(t) = V^set if Vfe : t 6 {t\,t\ + r re f] 




Exact integration [61] with temporal resolution dt 

Initial membrane-potential distribution at t = 0: uniform between and 9 


E 




Synapse 


Type 

Description 


Current synapse with 5-shaped postsyn. currents (PSCs) 

Input current of neuron i: Ii(t) = C m J2j Jij J2i S(t — tj — d) + Ii^xt(t) 

Static synaptic weights (see Connectivity) 



E 



Type 

Description 



Input 



uncorrected Gaussian white noise R m Ii ext (t) = /x ext + \fr^ii(t) (for i e 

mean /x ext = R m E t [J ijext (*)] f auto-correlation R 2 m E t [I iiext (t)I ji&(t (t + r)] = 
/4t + V 2 r m SijS(r) 

in discrete time t G {n-dt\n e N}, £(n-dt) piecewise constant within time 
interval dt, value drawn independently for each time point from a normal 
distribution with zero mean and standard deviation 1/y/dt 



Table 1. LIF network: Model overview 
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A 


Connectivity 


Name 


Value 


Description 


K 


1250 (inhibitory network) 


in-degree 




1000 (E-l network) 


excitatory in-degree 


e 


0.1 


network connectivity 


ry — ]Yf. /AT 


1/4 (E-l network) 


relative size of inhibitory subpopulation 


B 


Neuron 


Name 


Value 


Description 




80 


membrane resistance 


r m 


20 ms 


membrane time constant 




2 ms 


refractory period 


Preset 


OmV 


reset potential 


9 


15 mV 


spike threshold 


C 




Name 


Value 


Description 


J 


0.2 mV 


EPSP amplitude 


9 


6 (E-l network) 


relative IPSP amplitude 


d 


0.1ms 


synaptic delay 


D 


Input 


Name 


Value 


Description 


Mext 


1.59 


mean external GWN input 


V 


0.39 


SD of external GWN input 


E Simulation 


Name 


Value 


Description 


T 


10 or 100 s 


simulation time 


dt 


0.1ms 


time resolution 



Table 2. LIF network: Parameters (default values) 



50 



